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RESEARCH REPORT 

Developing Content Knowledge for Teaching Assessments 
for the Measures of Effective Teaching Study 

Geoffrey Phelps , 1 Barbara Weren , 1 Andrew Croft , 1 & Drew Gitomer 2 

1 Educational Testing Service, Princeton, NJ 

2 Rutgers, The State University of New Jersey, Piscataway Township, NJ 


This report documents the development of assessments of content knowledge for teaching (CKT) as part of the Measures of Effective 
Teaching (MET) study, funded by the Bill and Melinda Gates Foundation. The MET study was designed to develop a set of measures that 
together serve as an accurate indicator of teaching effectiveness. The study was implemented during the 2009-2010 and 2010-2011 
school years with more than 3,000 teachers in 6 predominately urban school districts. A total of 5 assessments of CKT were developed, 
piloted, and then administered as part of the MET study. The CKT assessments focused on the content knowledge used in recogniz¬ 
ing, understanding, and responding to the content problems that teachers encounter as they teach a subject. In English language arts 
(ELA), 2 assessments were developed: 1 for teachers of Grades 4-6 and 1 for Grades 7-9. In mathematics, 3 assessments were devel¬ 
oped: 1 for teachers of Grades 4-5, 1 for Grades 6-8, and 1 algebra I. A total of 2,080 final assessments were administered to 1,718 
teachers in the 6 participating MET study districts. Assessment results for 194 teachers were excluded due to evidence that assess¬ 
ments were either completed together by 2 or more participants or that insufficient time was devoted to represent a good faith effort at 
answering the assessment questions. The final sample included 1,886 assessments. Assessment scores included both selected-response 
and constructed-response (CR) questions. We used information from item level statistics, including percent correct and biserial cor¬ 
relations, to systematically remove poorly performing items in order to improve assessment reliabilities. Item level statistics for each 
assessment are presented. Descriptive statistics and histograms indicate that participants are well distributed over the range of possible 
score responses. Assessments had moderate to strong levels of reliability, ranging from 0.69 to 0.83. 

Keywords Teacher content knowledge; assessment; teaching quality; English language arts; mathematics 

doi: 10.1002/ets2.12031 


The content knowledge for teaching (CKT) assessments described in this report were developed as part of the larger 
Measures of Effective Teaching (MET) study. The goal of this development effort was to create a set of assessments that 
measure the types of content knowledge used in teaching practice and then to examine how these CKT assessments are 
related to other measures of teaching effectiveness. To provide context for the assessment development, the report starts 
with a brief overview of the larger MET study and of the general theory of CKT that guided both the development of 
the assessment framework and assessment forms. The second section of the report describes the process of developing an 
assessment framework, assessment items, and the piloting and revision steps that led to the construction of final assess¬ 
ments (Appendices A-E). The third section of the report presents results from the administration of the assessments to 
teachers participating in the MET study and includes information on item performance, assessment scoring and reliabil¬ 
ity, and the general characteristics of participant scores. Appendix F contains item level statistics (i.e., percent correct and 
biserial correlations) for all items administered. 


Background 

Measures of Effective Teaching Study 

The MET study was designed to develop a set of measures that together serve as an indicator of a teacher s impact on 
student achievement. The study was implemented during the 2009 - 2010 and 2010-2011 school years with approximately 
3,000 teachers in six predominately urban school districts. To limit the need for extensive additional testing of students, 
the study focused on grades and subjects in which most states currently test students — Grades 4 through 8 mathematics 
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and English language arts (ELA) — plus three additional courses that serve as gateways for high school students — Grade 
9 algebra I, English, and biology 

The research study was led by more than a dozen academic institutions, not-for-profit institutions, and other educa¬ 
tional consultants. Data about the teachers practice and their students achievement were collected from a number of 
sources, including the following: 

• Measures of student achievement gains on existing state assessments and supplemental assessments designed to 
assess higher-order conceptual understanding. 

• Classroom observations, scored using multiple observation protocols, and teacher reflections on their practice. 

• Measures of teachers’ CKT (mathematics and ELA only). 

• Surveys of student perceptions of the classroom instructional environment. 

• Surveys of teachers’ perceptions of the working conditions and instructional support at their schools. 

The measures of teachers’ CKT in mathematics and ELA were developed during the first year of the study and were 
administered to teachers participating in the study during the second year. 

Content Knowledge for Teaching 

Content knowledge is a long-established basic prerequisite for teaching a subject, and it is an essential requirement for 
teacher certification (Hill, 2007). However, scholars have argued that teachers need to develop forms of content knowledge 
that go beyond basic content proficiency to be effective in the classroom. The idea that teachers need to understand and use 
content in ways particular to teaching was captured in arguments developed by Shulman, who conceptualized pedagogical 
content knowledge (PCK) as comprising 

the most useful forms of representation of those ideas, the most powerful analogies, illustrations, examples, 
explanations, and demonstrations — in a word, the ways of representing and formulating the subject that make it 
comprehensible to others.... Pedagogical content knowledge also includes an understanding of what makes the 
learning of specific topics easy or difficult: the conceptions and preconceptions that students of different ages and 
backgrounds bring with them to the learning of those most frequently taught topics and lessons. (Shulman, 1986, p. 9) 

The notion of PCK has been influential to the education field. These ideas have contributed to research that has focused 
on the central role of content in teaching and that has provided a way to conceptualize teaching as professional work with 
its own unique knowledge base (Ball, Thames, & Phelps, 2008, p. 392). 

Ball et al. (2008) proposed a domain argument that includes the full range of content knowledge used in teaching 
a subject, with knowledge falling under two major categories: content knowledge and PCK. PCK includes three types of 
content knowledge that are a mix of content knowledge and knowledge of students, teaching, or curriculum. Knowledge of 
content and students includes, for example, knowledge of particular content misconceptions or confusions that students are 
likely to exhibit. Knowledge of content and teaching includes knowledge of the productive ways of representing particular 
content to meet student learning needs. Knowledge of content and curriculum includes knowledge such as how to use 
existing curricula to teach specific content. 

Content knowledge for teaching also includes three categories of content knowledge that do not involve knowledge 
of students, teaching, or curriculum. Common content knowledge is defined by its shared use in common across adult 
pursuits other than just teaching. For example, engineers and mathematics teachers both use knowledge of how to solve 
algebraic problems in conducting their day-to-day work. Horizon content knowledge is defined by knowledge of how 
different content ideas are connected across the content domain, often with more elementary or basic ideas connected 
to more complex or advanced ideas. For example, one would not expect a high school geometry course to include non- 
Euclidean geometry, but it is helpful for a teacher teaching the triangle sum theorem to know its connection to non- 
Euclidean geometry, as well as the mathematical importance of that connection. Specialized content knowledge includes 
types of content knowledge that are only used in teaching. For example, while someone skilled in mathematics may use a 
particular mathematical strategy or method to solve a set of problems, the teacher must understand a range of strategies 
that could be used to represent the math in different ways to address different types of student learning needs, differences in 
the mathematics involved in each of the strategies, and whether such strategies are mathematically valid and generalizable. 
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These are examples of a type of pure mathematics that are only needed for and used in the work of teaching mathematics 
and that do not require the additional knowledge about students or content representations that comprise PCK. 

Emerging theories about what constitutes content knowledge for teaching have received interest from teacher educa¬ 
tors, policy makers, and teachers themselves, because these ideas draw a direct connection between the work of teaching 
a subject and the content knowledge needed in order to carry out this work. While clearly teachers need to understand 
the content that their students are learning, evidence that teaching this content can require forms of content knowledge 
that go far beyond more conventional content proficiency strengthens arguments that teaching is professional work with 
its own unique professional knowledge base. These ideas also point to the potential and need for assessments that capture 
forms of content knowledge used in teaching that differ from what is measured on conventional knowledge tests. 

Developing a Design Framework and Content Knowledge for Teaching Assessments 
A Design Framework for Content Knowledge for Teaching Assessments 

Building on the literature, we define CKT as the content knowledge used in recognizing, understanding, and responding 
to the content practices that teachers engage in as they teach a subject. While much of the literature on content knowledge 
for teaching has focused on distinctions in the types of knowledge used in teaching, we place the emphasis instead on 
teaching itself and seek to identify the main ways that content knowledge is used in teaching practice (Ball & Bass, 2003). 
This focus on teaching practice is an important component of current theories of CKT. Domains such as PCK and its 
subcomponents are defined by the ways in which content is encountered and worked within teaching. In the classroom, 
PCK exists because teachers work with students’ content confusions or with content representations in teaching content. 
It is the work of teaching and how content is encountered in teaching that distinguishes domains of CKT from content as 
it is used outside of teaching practice. However, in our work to develop measures of CKT, we have shifted the emphasis 
in both conceptualization and actual question design from distinctions that differentiate knowledge type to distinctions 
that differentiate how knowledge is used in teaching. Instead of seeking to develop and design items that measure one 
type of CKT rather than another, we focus on the practices or tasks of teaching that make up the work of teaching subjects 
such as ELA and mathematics and organize our domains around those tasks. As a consequence, the assessment questions 
organize around the work of teaching in whatever mix of the various components of CKT that are needed to address the 
content problems that teachers need to recognize, understand, or respond to in a particular task. 

Our perspective in assessing CKT is defined by the recurrent practices that make up the work of teaching subjects 
such as mathematics and ELA. These recurrent practices, known as tasks of teaching, define what knowledge is needed for 
teaching and, thus, form a design framework for CKT assessments that directly links content knowledge to teaching prac¬ 
tice. One could label this a practice-based approach, because it takes instructional practice as the basis for both organizing 
knowledge and for designing assessment questions that can measure the content knowledge used in teaching. 1 

The CKT assessment design framework focuses on the content practices that teachers encounter in the range of settings 
and roles that define their work. Most obviously, this involves teachers’ interactions with students around the content and 
learning activities that occur within the classroom. However, teachers also engage in many content practices outside of 
the classroom as they consider or plan how to teach a topic, make sense of student work products to decide what to do 
next, share teaching with colleagues, design lessons or curriculum materials, interpret standards, prepare students for 
state tests, and much more. A practice-based framework for assessing CKT must sample from the full range of content 
practices that define that work of teaching a subject. 

These core practices, or tasks of teaching , are the organizing foundation for the CKT assessment design framework. 
Many tasks of teaching are common across subject areas and grade levels and thus, create a common structure that in 
turn provides a basis for elaborating the tasks of teaching encountered in a specific subject area and grade level. These 
elaborated descriptions of tasks of teaching provide the basis for defining the content problems that teachers need to 
recognize, understand, and respond to and that are the target of particular CKT assessment questions. 

The design framework developed for the MET study is outlined in Table 1 and describes the content-specific tasks 
that comprise the work of teaching at three levels: first a general level that is shared across subjects; next, a level of 
tasks that are specific to defined content domain; and finally, a level that is defined by integration of the actual con¬ 
tent topics that are taught to students with the work of teaching that content. At the highest level, represented in the 
two left-most columns, the content tasks of teaching are general. Tasks of teaching at this level include, for example, 
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anticipating student difficulties, evaluating student work, and explaining concepts. At this level, tasks of teaching are 
applicable to all subjects regardless of topic or grade level and, thus, provide a general organization that can guide 
development for any assessment of CKT. Tasks of teaching that are specific to a subject are indicated in the final two 
columns. 

The third level of the hierarchy specifies the content topic level distinctions that are the focus of each of the tasks of 
teaching. For example, a task of teaching, such as anticipating student challenges, misconceptions, partial misconceptions, 
alternate conceptions, strengths, interests, capabilities, and background knowledge (see Table 1), might focus on antic¬ 
ipating likely student misconceptions around number concepts, rational numbers, proportional reasoning, algebra, and 
so forth. The third level of the framework represents the particular ways that teachers work with the topical content that 
they are responsible for teaching. We have not attempted to represent this third level of the framework in Table 1 because 
of the level of detail that would be involved. In an ideal form of test development, all of the ways that content intersects 
with tasks of teaching would be identified and there would be a clear mapping between this elaborated map of content 
teaching and the test items that sample from this domain. This level of domain development was not undertaken for the 
MET assessments. However, the actual items, which are presented in the appendices, provide a useful illustration of how 
tasks of teaching involve work with particular topical content and the range of topical content that was addressed at the 
third level of the framework. 

The CKT assessments, and in particular the third level of the CKT framework, are guided by attention to general topic 
level strands that define the content that is taught and learned at the grade levels that are assessed. As a practical matter, 
identifying these general topic strands can present challenges in developing a valid and relevant assessment. Often states 
and even districts have quite different standards for the student content that is taught, the grade level at which this content 
is taught, and to what level of content proficiency. These differences make it difficult to justify the specific content that 
should be emphasized in a CKT assessment. It is worth noting that recent changes in student standards toward a more 
commonly shared set of student standards (e.g., the Common Core State Standards [CCSS]) could have a major impact 
on the development of CKT assessments of mathematics and ELA. If, for example, CCSS were fully adopted by all schools 
and states in the United States, then the CCSS would become the basis for the third level of the hierarchy. At this point, 
however, no such shared basis exists, and the third level of the hierarchy is by necessity a set of decisions and compromises 
in emphasizing content that is widely shared enough to be deemed a necessary component of CKT for a particular subject 
and grade span. 

The list of tasks of teaching (see Table 1) is not taken to be comprehensive, although we have made efforts to generalize 
across content domains. This list is not intended to imply that this is all teaching consists of, but it is a result of our 
focus on the content problems teachers must contend with in the work of teaching. The tasks on the list are not intended 
to be seen as entirely exclusive of one another. In many cases, the work of teaching might include multiple tasks, or a 
single task might potentially be described in more than one way. For example, a piece of student work completed in class 
might first be evaluated by the teacher, then become an example that is evaluated and selected because it highlights a 
particular point that the teacher has in mind due to anticipating certain misconceptions about the content. Larger tasks 
of teaching not represented here may be made up of smaller tasks of teaching. Responding to students, for example, is a 
combination of evaluating what the student has said and done, choosing an appropriate response, and then delivering that 
response. The tasks of teaching listed here would include evaluation of the student work and creation of an explanation, 
example, model, representation, or something else as appropriate to the student understandings or misunderstandings 
at hand. 

Tasks of teaching are the organizing basis for the design framework because each task represents a natural unit 
of teaching practice that has been identified in research or other relevant analyses of teaching practice as being 
a critical part of teaching. As natural units of practice, they often represent the coordination of different knowl¬ 
edge domains of CKT. For example, a task of teaching could simultaneously draw on common content knowledge, 
specialized content knowledge, and knowledge of content and teaching. Rather than artificially segmenting instruc¬ 
tional problems to align with a discrete knowledge domain of CKT, the tasks of teaching attempt to capture how 
teachers actually work with content in their day-to-day and moment-to-moment practice. This supports validity 
claims that assessments built on this framework measure the intended knowledge and reasoning and also increase 
the face validity of assessment tasks by situating them in contexts that are perceived by teachers as authentic and 
relevant. 
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Table 2 Advisory Group Members 


Consultant 

Affiliation 

Content area 

Margarita Calderon 

Johns Hopkins University 

English language arts 

Pam Grossman 

Stanford University 

English language arts 

Carol Lee 

Northwestern University 

English language arts 

James Marshall 

University of Georgia 

English language arts 

Deborah McCutchen 

University of Washington 

English language arts 

P. David Pearson 

University of California, Berkeley 

English language arts 

Deborah Ball 

University of Michigan 

Mathematics 

William Bush 

University of Louisville 

Mathematics 

Heather Hill 

Harvard University 

Mathematics 

Erik Jacobson 

University of Georgia 

Mathematics 

Rebecca McGraw 

University of Arizona 

Mathematics 

Judit Moschkovich 

University of California, Santa Cruz 

Mathematics 

Denise Spangler 

University of Georgia 

Mathematics 

Jon Star 

Harvard University 

Mathematics 


Assessment Development 
Development Teams 

The practice-based approach that led to the definition of the CKT construct was also the basis for developing assessments 
of CKT. For each subject area, a development team was created to generate items that were directly linked to the frame¬ 
work and, thus, to teaching that subject. Each development team comprised a mix of individuals, including professional 
assessment developers at Educational Testing Service (ETS), researchers working on the CKT project, teacher educators, 
and educators working directly with K-12 students. The goal of item development was to create assessment tasks that 
elicited the knowledge and reasoning used to recognize, understand, and act on content problems encountered in these 
tasks of teaching. 

Advisory groups were recruited to provide consultation and advice during the framework and item development. Mem¬ 
bers of the advisory committees (Table 2) were nationally recognized scholars in the field of mathematics or EL A teacher 
education. These consultants provided feedback and guidance during the early development of the CKT task of teaching 
framework. They were also given the opportunity to review and provide feedback on all of the items that were piloted in 
2010 . 

The development of the teacher knowledge measures was a joint effort of researchers from the University of Michigan 
(UM) and researchers and assessment specialists from ETS and was built directly on the extensive experience gained 
from the Learning Mathematics for Teaching (LMT) project at the UM. The proposed plan was to develop a series of six 
assessments of teacher knowledge for teachers of the following subjects and grades: 

• Mathematics Grades 4-5 

• Mathematics Grades 6-8 

• Algebra I 

• ELAGrades4-5 

• ELAGrades6-8 

• English Grade 9 

However, a series of discussions among the research team and advisors led us to decide to limit development to two ELA 
assessments, as the natural break was one in which the respective curricula emphasized skill development versus literature. 
Therefore, two ELA teacher knowledge measures were developed: one for teachers of ELA in Grades 4-6 and the second 
for teachers of ELA in the Grades 7-9. Three mathematics assessments were developed, as described above. During the 
project proposal phase, consideration was also given to developing a measure of content knowledge for teaching high 
school biology, but after discussions with the MET leadership, this was not deemed to be feasible given the constraints of 
the project budget, the short development timeline, and limited expertise in this area. 

The assessments of teacher knowledge were designed to be delivered over the Internet, allowing teachers to take 
assessments when they chose. From the beginning, we recognized the challenge of finding a reasonable balance between 


ETS Research Report No. RR-14-33. © 2014 Educational Testing Service 


9 





G. Phelps etal. 


Developing Content Knowledge for Teaching Assessments 


Ms. Hupman is teaching an introductory lesson on exponents. She wants to give her students a 
quick problem at the end of class to check their proficiency in evaluating simple exponential 
expressions. Of the following expressions, which would be least useful in assessing student 
proficiency in evaluating simple exponential expressions? 

A) 3 3 

B) 2 3 

C) 2 2 

D) All of these are equally useful in assessing student proficiency in evaluating simple 
exponential expressions. 

Figure 1 Sample mathematics content knowledge for teaching (CKT) item. 


Ms. Rice begins a unit on memoir writing by reading a passage from a literary model. She then 
asks students to complete a warm-up activity to help them generate ideas for their own writing. 

For each assignment, indicate whether or not it will help students focus their brainstorming on 
generating a memoir. 



Will help focus 
brainstorming 

Will not help focus 
brainstorming 

(A) Write a poem about the ways you have 
changed, using the form “I used to be.. .but 
now I am...” 



(B) Write a sequence of sentences describing 
some of your experiences, beginning each 
sentence with the phrase “I remember.” 



(C) Write a few adjectives that describe your 
personality. 



(D) Write down some of your favorite foods and 
describe what you like about them. 




Figure 2 Sample English language arts (ELA) content knowledge for teaching (CKT) table item. 


assessment length, participant burden, and adequate assessment reliabilities. Given that there were no stakes associated 
with assessments in the context of the MET study and participation was voluntary, the assessment length was largely 
driven by a decision to keep the assessments relatively short to reduce participant burden. If these teacher knowledge 
assessments had been intended to be used to discriminate among teachers for the purpose of high stakes decisions such 
as tenure and firing, then very high levels of assessment reliability and associated precision for individual classifications 
would be necessary. Achieving high levels of reliability is typically accomplished by creating long assessments. However, 
reliabilities can also be enhanced by making use of performance data from pilot administrations to select high performing 
items and strike a balance in design that maximizes assessment reliabilities and minimizes participant burden. For these 
particular assessments, our goal was to create assessments that had reliabilities above 0.70 and could be completed in an 
hour or less. 


Question Types 

The question development work built directly on the experiences of the LMT project at the UM. The LMT measures 
comprise two types of questions: single-selection multiple-choice, in which the single best answer is chosen from among 
the four answer choices given, and what we refer to as table questions , in which a common stimulus is presented with two 
or more answer choices (presented in columns), and a separate response must be given for each scenario presented in 
the rows of the table. Typically, table items present a yes or no response option. Likewise, the majority of the questions 
included in the MET assessments of teacher knowledge used these same two response types. Sample MET questions are 
shown in Figures 1 (example of a single-selection multiple-choice question) and 2 (example of a table question). 

For the purposes of clarity, in all subsequent descriptions of the assessments and in the discussion of the data analyses, 
each response to a selected-response question will be referred to as an item. Thus, as can be seen in Figures 1 and 2, a 
single-selection multiple-choice question corresponds to a single item, but a table question corresponds to several items, 
one for each row in the table (e.g., the sample table question corresponds to four items). Each table question in the MET 
teacher knowledge assessments corresponds to three to six items. (A discussion about the independence of the items 
within a table question is included in the scoring results section below.) 
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In addition to the two types of selected-response questions described previously, four of the five measures of teacher 
knowledge (i.e., all except algebra I) also included a small number of open-ended or constructed-response (CR) ques¬ 
tions. One compelling reason to include CR questions in assessments like these is that they can capture evidence of 
certain types of knowledge (e.g., generative reasoning) that is not available from selected-response questions. Although 
many large-scale assessments (teacher and student) have not included CR tasks because of the significant costs associated 
with scoring the tasks with human raters, recent developments in natural language processing have made it possible to 
conceive of scoring short constructed responses using automated scoring technologies. Thus, the proposal for the devel¬ 
opment of teacher knowledge measures included small-scale efforts to (a) develop CR questions that reliably measure 
some aspect of the CKT construct and (b) investigate the feasibility of using automated scoring technologies to score the 
responses. (Since the algebra I assessment was only administered to teachers at one grade level, Grade 9, the expected 
sample size was too small to support building automated scoring models, and thus, no CR items were included in the 
algebra I assessment.) 

Item-Writing Procedures 

This assessment development was not a standard effort in two important ways. First, for this study, the assessment 
development teams were simultaneously working within and iteratively developing a theoretical framework for the 
assessments. In more traditional assessment development, the assessment design and specifications emerge from a 
consensus-building process among content-expert development committee members. These specifications often take 
the form of lists of important content topics, rather than an overarching framework, and are defined prior to actual 
assessment development work. Second, because the content domains for basic mathematics knowledge and reading 
ability are fairly well specified, development of tests of teacher knowledge have traditionally followed a rather straight¬ 
forward procedure, creating questions that focus on content topics that include, and to some degree encompass, the 
student curriculum for the grade span. This requires assessment developers to have knowledge of their subject area 
and expertise in constructing assessments but not necessarily deep understanding of teaching. However, the starting 
point for development of the MET CKT assessments was teaching practice itself, as defined by the tasks of teaching. 
Therefore, the assessment development required simultaneous attention to teaching practice, teacher content knowledge 
as it is used in teaching practice, and more standard conceptions of the content knowledge represented by the student 
curriculum. Not only does this complicate the work because additional, deeper knowledge is required, but it also 
requires conventionally trained assessment developers to learn how to attend to dimensions of teaching in assessment 
development. 

We found that to do this work, assessment development teams needed a solid grounding in instruction in their subject 
matter area, and both needed to develop the skills to examine particular tasks of teaching and to identify the knowl¬ 
edge entailed by those tasks. Consequently, the mathematics and ELA assessment development teams each comprised 
researchers whose work focused on the study of teaching, subject matter experts, teacher educators, and assessment spe¬ 
cialists, many who also had teaching experience. The development teams also collaborated with external groups with 
specific expertise around teaching, such as a group of professional writers who both work directly with students and 
provide professional development to teachers of creative writing. 

Both of the development teams met regularly, virtually, and face to face, throughout the first year of the study to develop 
a pool of questions that could be used in piloting and as examples to generate additional questions. Members of the 
development teams drafted questions that focused on the tasks of teaching and core content topics for each particular 
assessment. All questions were reviewed and discussed in detail multiple times by the development teams to ensure that 
the questions would elicit the desired evidence of teacher knowledge and to inform the continuous development of the 
CKT framework. Questions that were deemed acceptable by the development teams were then reviewed for compliance 
with editorial standards and best practices for assessment development. All revisions to the questions were documented 
and tracked to enable us to study the development process at a future time. 

Pilot 

A pool of more than 200 questions was developed across the five measures. During the summer of 2010, two forms of 
each of the five assessments were assembled and piloted with a convenience sample of practicing teachers, teaching the 
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Table 3 Summary of Question Types, Item Exclusion, and Assessment Reliability 


Assessment form 

Total SR items 

SR items excluded 

SR item reliability 

Total CR items 

Final reliability 

Mathematics 4-5 

38 

0 

0.74 

2 

0.76 

Mathematics 6-8 

46 

0 

0.82 

2 

0.83 

Algebra I 

37 

2 

0.77 

0 

0.77 

ELA 4-6 

51 

5 

0.71 

2 

0.74 

ELA 7-9 

48 

3 

0.66 

2 

0.69 


Note. SR = selected response; CR = constructed response; ELA = English language arts. 


appropriate subject area and grade level. Extensive efforts were made to recruit potential participants for the pilot admin¬ 
istration of the MET CKT items. More than 13,000 teachers were sent invitations to participate. Potential participants were 
recruited in numerous ways, including direct individual contact using information provided by supporting school dis¬ 
tricts; charter school operators; and relevant organization LISTSERVs, newsletters, and Web sites. Approximately 1,600 
teachers consented to participate in the pilot study, with a total of 1,751 assessments completed. Those teaching at the 
elementary level were given the option to take both mathematics and ELA assessments. 

Participants in the MET study were not allowed to take the pilot assessments. The pilot assessments, along with a 
short demographic survey, were administered using the same Web-based assessment delivery platform that was used 
for the MET study administrations. Each assessment was designed to take approximately 1 hour to complete and was 
administered to 150-200 participants. Detailed analyses were then conducted of the item performance to examine item 
difficulty, discrimination, and latency. 

In parallel with the piloting, a small number of cognitive interviews were conducted with teachers (mostly from 
New Jersey), using a subset of the questions. These interviews were designed to elicit teachers thinking as they worked 
through the questions and to provide information about potential sources of confusion and/or construct-irrelevant vari¬ 
ance. Interviews were used to show whether and how effectively the intended constructs were actually being measured 
by the assessment questions. The results of these interviews were used to support the revision and refinement of the CKT 
questions included in the assessment forms for the study administration. 

Also in parallel with the pilot administration, all items were provided to an external advisory committee for review and 
comment. 

Data from the psychometric analyses and cognitive interviews, along with comments from the external reviewers, were 
used by the development teams to select and revise the questions for inclusion in the assessments administered to the MET 
study participants. The assessments as administered are summarized in Table 3, with the actual questions administered 
on each form presented in Appendices A-E. 

Final Assessments 

Assessment reliability (Cronbachs raw alpha) ranged from 0.69 to 0.83 for the CKT assessments based on selected- 
response and CR items (Table 3). Certain selected-response items were excluded from scoring and item analysis through 
a process elaborated in the Item Analysis and Assessment Reliability section of this report. 

A number of questions appeared on multiple assessments. Eight questions appeared on both ELA assessments. Six 
questions were common to the mathematics 4-5 and mathematics 6-8 assessments. Four questions appeared on both 
the mathematics 6-8 and algebra I assessments. One question was common to the mathematics 4-5 and algebra I 
assessments. As discussed in the results, these common items provided an opportunity to investigate differences in item 
performance across assessments and groups of teachers. 

Administration and Sample 

The administration of MET assessments of teacher knowledge occurred during the 2010-2011 school year. We admin¬ 
istered 2,080 CKT assessments to teachers among six districts participating in the MET study. Teachers received unique 
URLs to access and complete assessments online at any time during the administration period. The assessment for teach¬ 
ers of mathematics in Grades 4 and 5 was administered in fall of 2010, and the other four assessments were administered 
in early 2011. Assessments were administered to teachers at the MET-specified grade level (i.e., Grade 4 and Grade 5 
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Table 4 Exclusion Codes by Assessment 


Assessment 



Math 4-5 
n = 465 
n (%) 

Math 6-8 
n = 398 
n (%) 

Algebra I 
n= 148 
n (%) 

ELA 4-6 

n = 635 
n (%) 

ELA 7-9 

n = 434 
n (%) 

Total 
n = 2,080 
n (%) 

Exclusion category 

Missing responses 

0(0) 

3(1) 

0 

2 (<1) 

4(1) 

9 «1) 

Latency/response pattern 

8(2) 

5(1) 

3(2) 

13(2) 

4(1) 

33 (2) 

Unusual agreement 

60(13) 

17(4) 

2(1) 

65 (10) 

8(2) 

152 (7) 

Final scored sample 

397(85) 

373 (94) 

143 (97) 

555 (87) 

418(96) 

1,886 (91) 


Note. ELA = English language arts. Percentages shown are with respect to the assessment total n. 


mathematics teachers received the mathematics 4-5 assessment). The algebra I assessment was administered only to 
Grade 9 algebra I teachers. 


Excluding Cases From the Scored Sample 

We reviewed the assessment results in order to identify those that could not be used to generate reliable scores (e.g., cases 
of incomplete assessments in which responses were given only to the first few questions or cases in which the entire assess¬ 
ment was completed within a few minutes). In addition, as part of the review process, we noticed patterns in the timing 
data that suggested possible problems, including collaboration among groups (either pairs or larger groups) of test takers. 
We then conducted a systematic and more detailed analysis of assessment data. As a result, we excluded 194 assessment 
results from the final sample. These cases were not included in item analyses and were not individually scored. Cases were 
excluded for a variety of reasons, but the largest number of exclusions were cases of unusual agreement, instances in which 
two or more teachers appearing to have collaborated in ways that rendered it impossible to attribute the scores to a par¬ 
ticular individual. We excluded additional cases for other reasons that created concern about good faith effort, including 
questionable response patterns and response times that were not sufficiently long. The excluded cases are summarized in 
Table 4. 

The following descriptions address each of the rules that we applied for this set of analyses. While the overall exclusion 
rate is approximately 9%, exclusions were more frequent for elementary level assessments. 

Missing Responses 

In these cases, the teacher submitted fewer than 75% of required item responses to complete the assessment (i.e., 25% 
or more of that teachers responses were missing). In nearly all cases, these were sequential responses at the end of the 
assessment, as teachers were required to enter a response in order to proceed. Thus, these individuals exited the assessment 
prior to completion. 

Latency and Response Pattern 

Teachers completed the assessment at a rate deemed to be too fast to have considered items and thought about response 
choices. Assessments for which the total latency was less than 5 minutes or for which either half of the assessment was 
completed in less than 2 minutes and 30 seconds were excluded. Time spent on CR items was not included in this criterion. 

We checked each teachers set of responses for predominant submission of any single response option that suggested 
that items were not given serious consideration. If over 80% of overall responses or the entire second half of an assessment 
were selected as a single response option (e.g., the second choice for each item was selected), then the assessment results 
were flagged. Latency data were used as confirmatory evidence for these types of response patterns. All of these cases were 
also flagged for speedy assessment completion. 

Unusual Agreement 

We used statistical analysis to compare all possible pairings of teachers in the entire sample to detect unusual agreement. 
The methodology for detecting unusual agreement was developed at ETS as a means of test security; the numerical value 
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generated is known as the K-index (Holland, 1996; Lewis & Thayer, 1998). The software matched all possible pairs of 
teachers’ incorrect response choices on multiple-choice items. The K-index is 

an estimate of the chance that at least the observed number of matching incorrect responses would occur if the two 

test takers being compared were working independently on the section of the test being analyzed. If this probability is 

sufficiently small, it is taken as an indicator of “unusual agreement” between the two sets of incorrect responses being 

compared. (Lewis 8c Thayer, 1998, p. 1) 

Other characteristics of this detection method include: 

• Matched correct responses are not included in these analyses because they may indicate that respondents indepen¬ 
dently know the content and, thus, provide weak evidence of unusual agreement. 

• Omitted responses are ambiguous and are not considered evidence for unusual agreement. 

• In order to be able to conduct these analyses, there is a minimum number of multiple-choice items for which 
response data are available (each of the five CKT assessments met this minimum threshold). 

• Item difficulty is not controlled for in this analysis. 

For this analysis, if the K-index for a pair of assessment results was less than .01, then those results were flagged for 
unusual agreement. As described below, flagging by itself was not sufficient to exclude a score. Given a large sample 
size and all possible pairs of teachers for each assessment, many pairs would be identified by chance. Therefore, we col¬ 
lected further evidence to confirm unusual agreement between teachers, identified in pairs, whose results had a K-index 
below .01. 

A. Same district: The teachers in a pair are in the same district, indicating they have the opportunity to complete the 
assessment together. 

B. Same school: The teachers in a pair are in the same school, indicating they have the opportunity to complete the 
assessment together. 

C. End time: The teachers in a pair completed the assessment within 10 minutes of each other. 

D. Similar answers on multiple-choice questions: Either teacher in an identified pair must not differ in more than two 
incorrect responses for multiple-choice questions. 

E. Similar answers on table questions: The K-index is determined only using the multiple-choice items. In this analysis, 
we also checked the consistency of responses on the table questions. Teachers in an identified pairing must not 
differ in more than two incorrect table item responses. This criterion provided confirmatory evidence of unusual 
agreement. 

In summary, if the K-index is below .01 and criteria A and one of B, C, D, or E are satisfied, we excluded that case on 
the basis of unusual agreement. 

Recognizing that the K-index analysis has weak power when applied to pairs of test takers with few incorrect items, 
we also reviewed all of the assessment results for teachers in the same schools. We compared their assessment end times, 
scores, and latencies for the entire assessment and scores and latencies for each half of the assessment. Four pairs of teachers 
who completed ELA assessments were flagged for completing the assessment at the same time (within 5 minutes of each 
other) with matching responses and amounts of time spent per item. These teachers were not identified by the K-index 
because each teacher had only one incorrect response to selected-response items. On the basis of these other data, these 
were judged to be four cases of unusual agreement, and the eight associated assessment results were excluded from the 
final analyses. 

In sum, 152 assessments were excluded based on the unusual agreement criterion, and the remaining 42 cases due to 
other criteria used to identify scores that were deemed to provide invalid information on the scored responses. 

Final Data Used for Scoring 

The final sample that could be used for scoring contains 1,613 individual teachers who completed 1,886 assessments (273 
teachers completed both a mathematics and an ELA assessment). Table 5 summarizes the administration by assessment 
and by district, respectively. 
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Table 5 Final Sample of Content Knowledge for Teaching (CKT) Assessments by District 


District 



Assessment 




Math 4-5 
n (%) 

Math 6-8 
n (%) 

Algebra I 
n (%) 

ELA 4-6 
n (%) 

ELA 7-9 
n (%) 

Total 
n (%) 

A 

104 (26) 

58 (16) 

18 (13) 

131 (24) 

66(16) 

377 (20) 

B 

95 (24) 

83 (22) 

59 (41) 

125 (23) 

119(28) 

481 (25) 

C 

85 (21) 

113(30) 

25 (17) 

141 (25) 

120 (29) 

484 (26) 

D 

19(5) 

0 a 

21 (15) 

13(2) 

21(5) 

74 (4) 

E 

94 (24) 

29 (8) 

20(14) 

116 (21) 

18(4) 

277(15) 

F 

0 a 

90 (24) 

0 a 

29 (5) 

74 (18) 

193 (10) 

Total 

397 

373 

143 

555 

418 

1,886 


Note. ELA = English language arts. 
Assessment not administered in this district. 


Measures of Effective Teaching Content Knowledge for Teachers Assessments Results 
Scoring Results 

We investigated different scoring models and weightings for some of the item types used in the CKT assessments. In this 
section, we discuss the considerations and decisions made and then present the final score distributions for each of the 
assessments. 


Selected-Response Items 

Two types of selected-response questions were used in the CKT assessments: multiple-choice questions and table ques¬ 
tions. Each multiple-choice question was given equal weight and counted as 1 point because each required a single selected 
response. 

We investigated various models of scoring the multiple-response table questions. We examined intra-item correlations 
(between the items within each table question) and found, for nearly all table questions, that items within a table question 
were not highly correlated with each other; that is, each item appeared to measure a discrete aspect of CKT. We concluded 
that each item (within a table question) should be counted as 1 point, as there was no evidence that item performance 
violated assumptions of local independence. Had there been evidence supporting a lack of local independence, we would 
have adopted an alternate weighting strategy so as not to inflate reliability estimates artificially. Each table question counted 
as 3 - 6 points, depending on the number of items within the table question. 

Before calculating final scores for selected-response items, we analyzed item level data (biserial correlations) and assess¬ 
ment reliability. We systematically removed weakly discriminating selected-response items from each assessment before 
calculating final scores. We describe the item exclusion process in more detail below. 

Constructed-Response Items 

To complete the analysis of the assessments of teacher knowledge, we calculated scores for CR items. For each item, 
we defined a set of concepts or propositions that represented a correct and complete response to the prompt. We then 
developed a set of scoring rules that matched the presence of some subsets of concepts with a score on the item. For 
example, if the correct and complete answer for a problem required the presence of concepts A, B, and C, then scoring 
rules were created to assign a score when A, B, and C were present (i.e., the highest score possible for the item), as well as 
scores for subsets of concepts in the response (e.g., A and B or B only). Human scorers had to evaluate whether each of 
the key concepts was present in the response. 

Similarly, the automated scoring engine determined the presence or absence of evidence of each of the scoring concepts 
by determining the extent to which the test-taker response had the same semantic meaning as any or all of the targeted 
concepts and then applied the respective scoring rules. A large number of prescored responses were used to build models 
of acceptable and unacceptable responses, and then the computer system used natural language processing techniques 
to interpret whether different phrasing had the same semantic meaning as a target concept. Varied degrees of success 
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Table 6 Summary of Constructed-Response (CR) Items 


Assessment 

Item sequence number 

Number of concepts 

Initial score range 

Quadratic weighted kappa 

Mean scale score b 

Mathematics 4-5 

20 

5 

0-2 

n/a a 

0.77 

Mathematics 4-5 

21 

2 

0-2 

0.93 

0.99 

Mathematics 6-8 

22 

5 

0-4 

0.87 

1.88 

Mathematics 6-8 

23 

6 

0-2 

0.77 

1.51 

ELA 4-6 

18 

4 

0-3 

0.68 

1.13 

ELA 4-6 

22 

14 

0-4 

0.83 

1.47 

ELA 7-9 

12 

4 

0-2 

0.87 

1.31 

ELA 7-9 

18 

5 

0-3 

0.70 

1.19 


Note. ELA = English language arts. 

a Item machine scored only, so a measure of interrater reliability (quadratic weighted kappa) cannot be computed. 

b Initial score range used to generate raw score by two human raters. An average of the two rater scores was then scaled between 0 and 

3 points to calculate a final score for constructed-response items. 


were achieved with the automated scoring of the CR items, with the general conclusion that it is possible to develop 
CR items that measure some aspects of CKT and can be reliably scored using automated scoring techniques. Significant 
additional research and development around these techniques and the applicability to measures of CKT is needed. With 
one exception, the reported data are based only on human scores (each response scored by two humans). The scoring of 
one mathematics item was based entirely on numerical criteria (no text or language evaluation) and was machine scored. 
The interrater reliability of human scorers was calculated using the quadratic weighted kappa. The reliability of the human 
scoring for all CR items was judged to be adequate for the purposes of this study (Table 6). 

We compared two scaling approaches of CR items to determine the degree to which they factor into final total scores. 
The first approach was to scale CR items at equal value with selected-response items. The second approach was to scale 
CR items on a 0- to 3-point scale, which is triple the value of selected-response items. We calculated correlations for each 
approach with total scores including selected-response items only. Correlations were 0.99 or greater using the single-point 
scaling approach and 0.97 or greater using the triple-point scaling approach. Factoring this result in with the amount of 
effort required to answer CR items, we decided to use the 0- to 3-point scale to calculate final scores. 

A summary of CR item scoring is provided in Table 6. 


Item Analysis and Assessment Reliability 
Item Level Analyses 

We applied the item level scoring methods for each item type to create total raw scores. For each item, we calculated 
percent of responses correct (i.e., p-plus value) and biserial correlations as measures of item performance. We investigated 
the impact on the reliability of each assessment when excluding items with poor discrimination, as measured by item 
biserial correlations (i.e., the relationship between respondents’ performance on one item and performance on the entire 
assessment). First, items, including table items, with negative biserial correlations were excluded. Second, items were 
eliminated one at a time, starting with the item with the lowest biserial correlation. If the assessment reliability (measured 
by Cronbach’s raw alpha) increased by 0.01 or greater when an item was excluded, then that item was removed from the 
subsequent score computations, and the process was repeated. If the alpha did not increase by 0.01 or more from the 
previous value following the exclusion of an item, then that item was not removed from the subsequent calculations, and 
the process of excluding items from that assessment was concluded. The number of excluded items by form is reported in 
Table 3. More detailed information about each item is noted in the tables in Appendix F. 


Including Constructed-Response Items in Scale Scores 

We investigated the impact on the reliability of each assessment of including CR items, as measured by Cronbach’s raw 
alpha. Assessment reliabilities improved by 0.01 or greater after including CR items. There was no change in the algebra I 
assessment reliability because it does not contain CR items. No CR items were identified as low-performing or in need of 
consideration for exclusion. 
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Table 7 Range of Item Difficulty 


Assessment 

N 

Minimum % correct 

Maximum % correct 

N < 25% correct 

N> 75% correct 

Mathematics 4-5 

Multiple-choice items 

14 

11.6 

73.3 

4 

0 

Table items 

24 

27.7 

96.0 

0 

8 

Mathematics 6-8 

Multiple-choice items 

14 

9.4 

67.6 

3 

0 

Table items 

32 

35.4 

98.1 

0 

16 

Algebra I 

Multiple-choice items 

19 

14.0 

76.9 

3 

2 

Table items 

18 

30.1 

100.0 

0 

11 

ELA 4-6 

Multiple-choice items 

17 

26.9 

91.9 

0 

4 

Table items 

34 

33.3 

96.0 

0 

17 

ELA 7-9 

Multiple-choice items 

16 

20.3 

81.6 

1 

2 

Table items 

32 

10.5 

99.3 

2 

18 


Note. ELA = English language arts. 



Table 8 Comparison of Percent Correct on 

Common Subsets of Content Knowledge for Teaching (CKT) Items 


Items shared 

Percentage of items with average scores 

Assessment forms 

across forms 

higher on higher-grade form 

Mathematics 4-5 and Mathematics 6-8 

9 

100 

Mathematics 6-8 and Algebra I 

12 

92 

ELA 4-6 and ELA 7-9 

19 

79 


Note. ELA = English language arts. 


The results from the item exclusion analysis and inclusion of constructed response items are summarized in Table 3. 
The resulting assessment reliabilities range from 0.69 to 0.83. 

Comparing Item Difficulty Across Assessments 

As the item level data suggest (Table 7), mathematics assessments were, in general, more difficult than the ELA assess¬ 
ments. No item on the ELA 4-6 assessment was answered correctly by less than 25% of respondents, and more than 40% 
of items on the ELA assessments were answered correctly by 75% or more of respondents. The mathematics 4-5 assess¬ 
ment contained the highest percentage of items (29%) that were answered correctly by less than 25% of respondents and 
the lowest percentage (21%) of items for which more than 75% of respondents answered correctly. However, the algebra 
assessment had proportions similar to the ELA assessments regarding items that were difficult or easy for the majority of 
participants. 

Comparing Item Difficulty Across Grade Levels 

Teachers at different grade levels will have varying experience in teaching particular concepts. It is also true that teachers 
in middle school tend to have been more likely than elementary teachers to major in the content area that they are teaching 
(Gitomer, 2007). Thus, we explored whether consistent differences existed in performance between these two groups on 
the same CKT items shared across tests taken by lower-grade and higher-grade teachers (e.g., mathematics 4-5 vs. math¬ 
ematics 6-8). We found a consistent pattern where a greater percentage of teachers at the higher-grade levels answered 
common items correctly compared with teachers at the lower-grade levels. This discrepancy in group performance is 
greater on common mathematics items than on ELA items (Table 8). 

Examining Score Distributions 

Overall, mathematics assessments were more difficult than the ELA assessments, as indicated by the mean percent correct 
scores reported in Table 9. 
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Table 9 Descriptive Statistics of Content Knowledge for Teaching (CKT) Assessment Scale Scores 


Assessment 

n 

Minimum 

Maximum 

M 

SD 

Mathematics 4-5 

397 

20.5 

93.2 

52.2 

14.4 

Mathematics 6-8 

373 

22.6 

94.2 

62.1 

14.9 

Algebra I 

143 

22.9 

97.1 

61.5 

14.3 

ELA 4-6 

555 

30.8 

89.4 

66.4 

11.7 

ELA 7-9 

418 

31.4 

88.7 

64.7 

10.5 


Note. ELA = English language arts. 



Final Score 


Figure 3 Mathematics 4-5 score distribution (% correct). 



Figure 4 Mathematics 6-8 score distribution (% correct). 


To further examine the distribution of these scores across the assessment samples, we present score distributions on the 
percent correct scale scores in Figures 3-7. The distribution of scores is near normal for all assessments with a very broad 
range of scores. The mathematics 4 - 5 assessment distribution is positively skewed with a large number of low scores, and 
the ELA distributions are negatively skewed with a large number of high scores. The other mathematics assessments do 
not appear to be significantly skewed. 


Relationship Between Performances on Both Elementary Level Assessments 

We designed the administration of the CKT assessments so that we could make comparisons across assessments. We ana¬ 
lyzed the performance of teachers who completed both assessments at the elementary level in ELA and mathematics. After 
removal of unreliable scores, there remained a total of 271 teachers who had valid scores for both elementary assessments. 
We found a significant correlation between performances on both assessments (r = 0.49, p < .01). This relationship is rep¬ 
resented graphically in Figure 8. It is also noteworthy that the ELA and mathematics assessments are clearly measuring 
different domains of knowledge, since the lack of correlation cannot be explained simply by uncorrelated measurement 
error (the correlation disattenuated for measurement error = 0.68). 

Table 10 summarizes the performance on the mathematics and ELA CKT assessments by quartile. While quartile rank 
is similar for many teachers across assessments, there is a substantial number of teachers who are relatively strong on one 
test and relatively weak on another. For example, of the 85 teachers in the lowest math quartile, 17 (20%) have ELA scores 
in the upper two quartiles. 
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Figure 5 Algebra I score distribution (% correct). 
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Figure 6 English language arts (ELA) 4-6 score distribution (% correct). 



Figure 7 English language arts (ELA) 7-9 score distribution (% correct). 


Summary 

The CKT development conducted as part of the MET study achieved a number of important goals. We highlight three 
accomplishments and findings: (a) the development of an assessment framework, (b) the generation of reliable assessments 
suitable for differentiating teachers across a range of CKT proficiency, and (c) practical considerations for how these 
assessments might be used to assess teacher quality. 

Prior to the MET study, the assessment of CKT was carried out as part of research projects, primarily in mathemat¬ 
ics, and largely without a well-articulated assessment design framework. One goal of the MET study was to propose a 
framework that could support identifying domains of CKT both within a given subject and across different subject areas. 
An important contribution is the CKT framework itself, along with the evidence supported by the development project 
that the framework can be used to support assessment development. The framework provides a starting point that can 
guide development efforts in other subjects and grade levels. The high-level categories provide a general organization that 
allows for making connections across different assessments. The lower-level, subject-specific categories provide examples 
that can guide the development of analogous or unique subject-specific tasks as needed. The CKT framework holds the 
promise of being generative for future assessment development. 

The CKT framework also focuses attention on teaching. By definition, assessments of CKT are focused on the content 
knowledge that is needed to recognize, understand, or act on the content problems encountered in teaching. For this 
reason, CKT questions typically incorporate a teaching scenario or setting that test takers must consider in answering the 
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Figure 8 Scores on elementary level assessment for teachers taking both the mathematics and English language arts (ELA) assessments. 


Table 10 Elementary Assessment Score Quartile Crosstabulation 


Math scale score quartiles 



ELA scale score quartiles 


Total 

Ql 

Q2 

Q3 

Q4 

Ql 

51 

17 

14 

3 

85 

Q2 

18 

23 

16 

8 

65 

Q3 

8 

16 

25 

16 

65 

Q4 

5 

12 

14 

25 

56 

Total 

82 

68 

69 

52 

271 


Note. ELA = English language arts. 


content problem. Organizing the CKT assessment framework around tasks of teaching provides direct guidance for the 
types of teaching contexts that need to be represented in the assessment tasks. Our success in using the CKT framework to 
develop assessment questions across five tests in ELA and mathematics demonstrates the utility of a framework designed 
around tasks of teaching. 

Another important contribution of this work is the actual set of assessments supported by the evidence that they can be 
used to generate reliable test scores that differentiate teachers across a wide range of the score distribution. The reliabilities 
for these five tests ranged from a low of 0.69 for Grades 7-9 ELA to a high of 0.83 for Grades 6-8 mathematics. While these 
reliabilities may seem low, it is important to realize that these assessments are comparable in their measurement quality to 
existing tests of teacher knowledge. The finding that the CKT tests for ELA were less reliable than for mathematics is also 
true for traditional content tests for teachers (ETS, 2010, p. 58), likely indicating domain difference rather than an inherent 
shortcoming of the CKT measures. Further, the CKT assessments are half the length of knowledge tests in current use. If 
the tests had been of equivalent lengths, the reliabilities would have been comparable to the more traditional knowledge 
tests, such as those from the PRAXIS® series. The CKT tests also provided adequate reliability across the wide range of 
ability. For all tests, a substantial number of teachers answered less than 50% of the items correctly and a substantial 
number answered more than 75% of the items correctly. This finding suggests that these tests would provide information 
on teachers’ CKT across a large range of the knowledge continuum. 

Finally, the results from the study also provide valuable information on how these assessments function in real school 
settings. The MET study involved teachers from six districts, a wide range of schools, and many different backgrounds. 
In a testing situation with no stakes attached, where teacher scores remained anonymous, and in which teachers were 
allowed to self-administer the assessment, evidence showed that the majority of teachers took these assessments seriously. 
Most teachers provided a good faith effort, spending the time needed to answer all assessment questions. This effort was 
true for teachers who received high scores and those who received low scores. This finding is encouraging, as it suggests 
that these assessments can be used to gather reliable information under conditions that do not include strong incentives 
for participation. 
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At the same time, a substantial number of teachers did not complete assessments as directed. On the elementary tests, 
nearly 15% of teachers appear to have completed the assessment with another teacher. This was the case even though 
teachers themselves were not being evaluated. It is unlikely that assessments such as these could be self-administered 
under conditions where teachers were being evaluated or other stakes were attached to their performance. In such cases, 
the number of collaborating teachers would likely increase along with the incentive to achieve a high score. Testing in 
situations where scores were public or used for consequential decisions would need to take place in a secure and proctored 
testing environment. 

The results of this development effort are an assessment design framework and a set of assessments that reliably mea¬ 
sures the types of content knowledge used in teaching practice. These assessments can be used to examine how measures 
of CKT are related to other measures of teaching effectiveness. They can be administered and studied in a variety of profes¬ 
sional contexts. Future development of these assessments, or the creation of additional ones, can improve how we measure 
teaching quality and contribute to student learning. 
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Anita O’Brien, Rob Rarich, Frank Rijmen, Melanie Schine, Jana Sukkarieh, Barbara Suomi, Amy Swauger, Sailesh Vezzu, 
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Notes 

1 The distinction between organizing assessment by tasks of teaching rather than types of CKT is rather subtle. For a full discussion 
of this shift of perspective and what this particular approach offers for assessing content knowledge, see Phelps, Howell, and Kirui 
(2013). 
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Appendix A 

Mathematics 4-5 Assessment 


Table A1 Item/Question Sequence Number for Mathematics 4-5 Assessment 


Item/question sequence number 

Answer key 

Item/question sequence number 

Answer key 

1 

A 

lOd 

Provides 

2 

C 

lOe 

Provides 

3a 

Not problematic 

11 

B 

3b 

Problematic 

12 

B 

3c 

Not problematic 

13 

B 

3d 

Not problematic 

14a 

Provides 

3e 

Problematic 

14b 

Does not provide 

4 

C 

14c 

Provides 

5 

A 

14d 

Provides 

6 

A 

14e 

Does not provide 

7a 

Provides 

15 

B 

7b 

Does not provide 

16 

B 

7c 

Provides 

17 

C 

7d 

Provides 

18 

C 

7e 

Provides 

19a 

Can 

8 

A 

19b 

Can 

9 

B 

19c 

Cannot 

10a 

Does not provide 

19d 

Can 

10b 

Provides 

20 

Open-ended response 

10c 

Does not provide 

21 

Open-ended response 


Task 

Question Number 

MET1 

1 of 21 


Assessment Tools 



Hannah s Work 


9j37n? 


-8,100 

900 9s 

249 


-ISO 

20 9s 

69 


-63 

7 9s 

6 

927 9s 


So $,349 


6 left over 
- 9 - 927R<5 


Alan s Work 


10 , 

10 '>927 
300 
300 

_300 I 

9JSJ4? 
-2,700 
5,649 
-2,700 
2,949 
-2,700 
249 

- 90 
159 

■_90 

69 

- 45 
24 


So, you get 927 
with remainder 6. 


Felipe s Work 

927 

9J57345' 

-SI 

24 

-IS 

69 

-63 

6 


The answer is 927^ . 


After Mr. Ikler’s class explored different strategies for dividing large 
numbers by single-digit divisors, they worked on the following 
problem. 


Divide 8.349 by 9. 

Students produced a variety of written strategies, some that are 
shown to the left. 

Which of these students used a strategy that could be applied to 
correctly divide any two whole numbers? 

O Hannah. Alan, and Felipe 
CD Hannah and Alan only 
CD Alan and Felipe only 
CD Felipe only 


Figure A1 Question#!. 
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Task 

Question Number 

MET1 

2 of 21 


Ms. Castle gave her students a task that included the following question. 

What value should be written where the arrow is pointing? 


When correcting their papers, she noticed that many students gave an incorrect answer of y instead of the correct 
answer of y. Of the following, which is the most likely reason for this error? 

CD The students did not correctly identify the interval from 0 to 1 as the whole. 

CD The students counted the number of tick marks instead of counting the number of spaces between the tick 
marks. 

CD The students counted the numbers of intervals without regard to the length of the intervals. 

CD The students incorrectly identified the relevant benchmarks for this problem. 



Figure A2 Question #2. 


Task 

Question Number 

MET1 

3 of 21 


Assessment Tools 


I ► II 

J _| Next \ \ 


Ms. Bowman found a supplemental worksheet on a Web site for teachers. The worksheet asked students to fill in one or more 
blanks in a mathematical statement with numbers that would make the statement true. However, she w as concerned that some of 
the exercises on the worksheet were mathematically problematic because the blanks could not be filled in with numbers that 
would make the mathematical statement true. For each of the following, indicate whether or not the exercise is mathematically 
problematic. 



Mathematically Problematic 

Not Mathematically Problematic 

8 + 15 =_ + 9 



14 + 5 = 19+ 5 = 24+ 5=_ 



o 

l 

N| 

u> 

+ 

1 



29 -_ = 22 + 6 = 28 



6-2=_+7 =_+ 5= 16 




Figure A3 Question #3. 
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Task 

Question Number 


Assessment Tools 

MET1 

4 of 21 


1 ► 1 


| Next | 


Ms. Hupman is teaching an introductory lesson on exponents. She wants to give her students a quick problem at the end 
of class to check their proficiency in evaluating simple exponential expressions. Of the following expressions, which 
would be least useful in assessing student proficiency in evaluating simple exponential expressions? 

o 33 
o 23 
o 22 

O All of these are equally useful in assessing student proficiency in evaluating simple exponential expressions. 


Figure A4 Question #4. 



Figure A5 Question #5. 
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Task 

Question Number 


Assessment Tools | 

MET1 

6 of 21 

I 

1 

;=> I 

»t« i 


In a unit on division, Ms. Jimenez gave her students the following problem. 

Carlos wants to cover the bottom edge of his window with a row of tiles that are each 5 inches long. If the 
bottom of the window is 42 inches long, how- many tiles will he need to buy? 

When the students finished, she wanted to give them another problem that was mathematically similar to the first. Of the 
following, w’hich is most similar to the original problem, both in terms of the meaning of division and the nature of the 
remainder? 

CD Tim needs to pack 42 binders into boxes in order to ship them. If each box can hold 5 binders, w hat would be the 
fewest number of boxes needed to ship all of the binders? 

O Gabriela has 42 stickers and wants to divide them up equally among 5 of her friends. How many stickers should 
each friend get? 

CD Robert wants to wrap gifts, and he has 42 feet of ribbon on a spool. If each gift requires 5 feet of ribbon, how 
many gifts can be wTapped using the ribbon on the spool? 

CD Teresa has 42 inches of string. If she cuts the string into 5 equal pieces, how* long will each piece be? 


Figure A6 Question #6. 


Task 

Question Number 


Assessment Tools | 

MET1 

7 of 21 

1 

1 

1 

► 1 

Next | 


In a unit on proportional reasoning. Ms. Richmond's class w r as discussing the following problem. 

If 4 cups of cocoa and 2 cups of sugar yield 16 brownies, how- many cups of cocoa and how many cups of sugar are needed to make 
24 brownies? 

Ms. Richmond's students used different strategies to solve the problem. For each strategy, indicate whether or not it provides evidence of 
mathematically valid student thinking. 



Provides 
Evidence of 
Mathematically 
Valid 

Student Thinking 

Does Not Provide 

Evidence of 
Mathematically 
Valid 

Student Thinking 

48 brownies need 12 cups of cocoa and 6 cups of sugar. To make 24 brownies, I need 6 cups of cocoa 
and 3 cups of sugar. 



4 and 2 both go into 16. 4 plus 2 is 6, half of 6 is 3, and 6 and 3 both go into 24. so you need 6 cups of 
cocoa and 3 cups of sugar to make 24 brownies. 



1 brownie needs ^ cup of cocoa and cup of sugar. To make 24 brownies, I need to multiply by 24 
for cocoa and sugar. Thus, I need 6 cups of cocoa and 3 cups of sugar. 



6 cups of cocoa and sugar makes 16 brownies, so 24 brownies need 9 cups of cocoa and sugar. Since the 
ratio of cocoa to sugar is 2:1,1 need 6 cups of cocoa and 3 cups of sugar. 



Since 1 cup of sugar is needed to make 8 brownies, I need 3 cups of sugar to make 24 brownies. The 
amount of cocoa is tw o times the amount of sugar in the recipe, so I need 6 cups of cocoa. 




Figure A7 Question #7. 
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Task 

Question Number 


Assessment Tools | 

MET1 

8 of 21 


1 

Z=> 1 

Net | 





Ms. Malloy's textbook suggests using white chips (for positive integers) and red chips (for negative integers) to teach 
integer subtraction. To solve the problem 2 - (-3). the book describes the process below. 

1. Start with two white chips. 

2. Add three pairs of red and white chips. 

3. Then take away three red chips for the -3. 

4. There are 5 white chips left, so the answer is 5. 

Ms. Malloy is concerned that her students will see that adding red and white chips sets up the subtraction, but not 
understand why the additional chips can be added. Of the following, which provides a mathematical explanation for why 
the three pairs of red and white chips can be added? 

CD By adding both a red chip and a white chip to an expression, you are really adding a zero, wiiich does not change 
the value of the original amount. 

CD The 2 is two white chips and the -3 is three red chips. When you add the three red chips, you also add three 
w’hite chips. That way. when you take away the red chips, the total number of white chips will be 5. which is the 
correct answer. 

CD The three red chips are for the -3 and the three white chips are w'hat the red chips become because the negative 
of a negative is positive. 

CD Red chips represent negative integers, but when you subtract negative three, you get positive three, so you need 
to add three w^hite chips as wrell as three red chips. 


Figure A8 Question #8. 


Task 

Question Number 


Assessment Tools | 

MET1 

9 of 21 








Mr. Chamberlain is concerned that his students' use of the calculator has led them to view the equal sign as a signal to 
carry out an operation rather than as a symbol indicating equality. Of the following missing-number problems, which 
would best assess whether students understand the mathematically correct meaning of the equal sign? 

O _ + _= 18 

O 7 + 5 =_+ 6 

O _= 17 + 9 + 5 
O 23+4 =_= 4 + 23 


Figure A9 Question #9. 
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Task 

Question Number 

MET1 

10 of 21 




asked to explain their reasoning. Of the following, which student responses provide mathematically valid explanations for why y is greater than 


9 ‘ 


For each student response, indicate whether or not it provides a mathematically valid explanation. 



Provides a 
Mathematically 
Valid Explanation 

Does Not Provide a 
Mathematically 
Valid Explanation 

When you compare them. y is greater than y because 7 is greater than 6. 



You can see that y is greater than y because ninths are smaller than eighths, which means that y is less than 

y which is less than y. 



You just need to look at how many pieces are missing. y is greater than y because y is only missing one piece 

from the whole, but y is missing three pieces from the whole. 



I think y is greater than y because y has more pieces than y and those pieces are larger. 



y is greater than y because y is equal to y . and because y is greater than y, y is farther away from 1 than 

7 . 

s“- 




Figure A10 Question #10. 


Task 

Question Number 


Assessment Tools 

MET1 

11 of 21 


1 

► 1 







To assess her students’ prior knowledge about evaluating arithmetic expressions. Ms. Santiago assigned a worksheet of 
problems. She noticed that Alexis answered the first two incorrectly and the next two correctly. 

1) 7 x 2 - 6 + 3 s: S' 

2) 9 — 5 + (16 t 8) c £ 

3) 9 + 24+3-1 -=./£, 

4) 17-(3+7x2) “ O 

Which of the remaining problems is Alexis likely to answer incorrectly? 

O 8 + 7-12-5-3 
O 13-3x2 + 5 
O (27 +3-4)+ 8 
O (16 ~ 12) x 5 + 10 


Figure All Question #11. 
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Task 

Question Number 


Assessment Tools | 

MET1 

12 of 21 


1 

1 ► II 

| Next | 



Ms. Jones's class has been studying area and perimeter. Some of the students in her class are learning English; they speak 
Spanish at home and when they work in small groups. One day they work on the following problem. 

• Look for all of the rectangles with area 36 (where length and width are integers). Write down the dimensions. 

• Calculate the perimeter for each rectangle. 

• Describe a pattern relating the perimeter and the dimensions. 

Of the following student responses, w hich most accurately describes the relationship between the dimensions of a 
rectangle and its perimeter? 

CD A rectangle has ... two ... short sides, and two long sides. The perimeter of the ... long ... skinny rectangle is big. 
It's about twice longer than the square. 

O The longer the ah ... the longer [traces the shape of a long rectangle with her hands several times] the ah ... the 
longer the. thing, you know' the more the perimeter, the higher the perimeter is. 

CD The rectangle ... the one with 4 width and 9 length ... its perimeter 26. and its area is bigger, it's 36. The rectangle 
3 width and 12 length ... it’s perimeter 30. It's 4 more ... it goes up by 4 ... higher. 

CD E ac ^ °f these student responses describes the relationship with equal accuracy. 


Figure A12 Question #12. 


Task 

Question Number 


Assessment Tools | 

MET1 

13 of 21 

1 


1 

•text | 




During an introductory unit on integers, Ms. Guerra asked her students to suggest a sentence illustrating what the word 
''opposite" meant to them. She w-anted to use one or two of their suggestions as images to build on as they learned about 
integers. Of the following student suggestions, which gives the LEAST accurate illustration of the mathematical meaning 
of the word "opposite" as it is used in the context of integers? 

O Profit is the opposite of loss. 

<D Entrance is the opposite of exit. 

CD Clockwise is the opposite of counterclockwise. 

O All of the student suggestions give equally accurate illustrations of the mathematical meaning of the word 
"opposite." 


Figure A13 Question #13. 
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Task 

Question Number 

MET1 

14 of 21 


Assessment Tools | 

□ 

1 ► 

| Next | 


Mr. Hinton gives his students the following 
division problem to solve. 

1056 -5- 12 

As he moves around the room, he notices his 
students using several different strategies. For 
each of the following student solutions, indicate 
whether or not the work provides evidence that 
the student is reasoning correctly about this 
problem. 



Provides Evidence of Correct 
Student Reasoning 

Does Not Provide Evidence of 
Correct Student Reasoning 

120 10 

720 50 

960 20 

1020 5 

1056 3 

SS 



1000 + 10 100 

100 -12 33 



1200 100 

120 _10 

1080 90 

24 2 

1056 SS 



960 + 12 SO 

96+12 S 

1056 + 12 SS 



1000 + 10 - 100 

56 + 2 - 28 

-100 -20 + S - SS 




Figure A14 Question #14. 



Figure A15 Question #15. 
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Task 

Question Number 


Assessment Tools 

MET1 

16 of 21 



◄ 1 

1 ► 1 

| Next | 




During a unit on decimals. Shanti claimed she knew an easy way to divide by ten. 


To divide by ten, you just move the decimal point one place to the left. For example: 427.3 -r 10 = 42.73. 


Ms. Bums asked her students to explain why Shanti's rule works. Which of the following student responses provides the 
best evidence that the student understands whv Shanti's rule works? 

O Shanti's rule works because if you multiply the answer by 10 it moves the decimal point one place to the right 
and you get the starting number. 

Shanti's rule works because dividing a number by 10 divides the value for each digit by 10, which is the same as 
moving all of the digits over one place to the right. 

<d> Shanti's rule works because w hen you divide you have to add the number of decimal places in the dividend and 
the number of ending zeros in the divisor—so with 10 it's one zero, which is one additional decimal place or just 
moving the decimal point one to the left. 

O These all provide comparable evidence that the students understand why Shanti's rule works. 


Figure A16 Question #16. 


Task 

Question Number 


| Assessment Tools 

MET1 

17 of 21 


1 

Reuieu> 




A lesson in Ms. Taylor’s textbook states the associative and commutative properties of addition. To motivate the students 
to learn the properties, she tells her students that the properties can often be used to simplify the evaluation of 
expressions. 

She wants to give her students an example that will focus their attention on how these properties can be useful in 
evaluating expressions. Of the following expressions, which would best serve her purpose? 

O (455 +456)+ (457+ 458 ) 

O (647 + 373 ) + ( 227+ 456) 

O ( 551 +775 ) + ( 49+ 225 ) 

O Each of these expressions would serve her purpose equally well. 


Figure A17 Question #17. 
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Task 

Question Number 

MET1 

18 of 21 


Assessment Tools | 


3 1 

Net | 


Ms. Schrader is teaching her class how to translate verbal expressions into algebraic expressions. Some of the students in 
her class speak Spanish at home and when they work in small groups. Of the following verbal expressions that translate 
into x - 5. which would be the most difficult for her Spanish-speaking students to translate correctly? 

CD A number minus 5 

CD The difference of a number and 5 

CD 5 less than a number 

CD Each of these expressions would present a similar level of difficulty for Spanish-speaking students to translate. 


Figure A18 Question #18. 


Task 

Question Number 


Assessment Tools | 

MET1 

19 of 21 


n „► 


Next 


Mr. Xavier is teaching his students to solve word 
problems. Students are learning to draw pictures 
to help them reason about the situations and about 
the computations used to solve the problems. Mr. 
Xavier wants to give his students problems where 
each problem has a figure and a computation and 
have his students discuss in small groups whether 
the figure can be used to model the given 
computation. For each of the following, indicate 
whether the figure can or cannot be used to model 
the given computation. 



Figure A19 Question #19. 
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Figure A20 Question #20. 


Task 

Question Number 


Assessment Tools | 

MET1 

21 of 21 

1 

» 1 

ReiMetuI 




Mr. Colwell gave his students the following problem. 




vvv\ 

//// 


The first figure is made up of 6 toothpicks. The second is made up of 10 toothpicks. Use the variable n to 
write an expression to show the number of toothpicks in any figure. 


Ian's expression was 4n + 2. 

Explain what the variable n in Ian’s response most likely represents. 


Samantha's expression was n + 4. 

Explain what the variable n in Samantha's response most likely represents. 


Figure A21 Question #21. 
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Appendix B 

English Language Arts (ELA) 4-6 Assessment 


Table B1 Item/Question Sequence Number for English Language Arts (ELA) 4-6 Assessment 


Item/question sequence number 

Answer key 

Item/question sequence number 

Answer key 

1 

B 

12d 

Would not 

2 

A 

12e 

Would 

3 

C 

13 

C 

4 

B 

14 

D 

5 

A 

15 

B 

6a 

Yes 

16a 

Effective 

6b 

Likely 

16b 

Effective 

6c 

Likely 

16c 

Effective 

6d 

Not likely 

16d 

Not effective 

6e 

Likely 

16e 

Not effective 

6f 

Not likely 

17 

B 

7 

C 

18 

Open-ended response 

8a 

Will 

19a 

Will help 

8b 

Will not 

19b 

Will help 

8c 

Will 

19c 

Will not help 

8d 

Will not 

19d 

Will not help 

9 

Accurate 

20 

B 

10 

D 

21 

C 

11a 

Accurate 

22 

Open-ended response 

lib 

Accurate 

23a 

Does not describe 

11c 

Not accurate 

23b 

Does not describe 

lid 

Accurate 

23c 

Describes 

lie 

Not accurate 

23d 

Describes 

Ilf 

Not accurate 

24 

A 

12a 

Would not 

25 

D 

12b 

Would 

26 

B 

12c 

Would 




Task || 

| Question Number 


Assessment Tools 

MET1 

1 of 26 


1 

| 

^ 1 

► 

Next 



Mr. Kowalski and his sixth-grade students are in literature circles discussing A Year Down Yonder, a historical novel about 
life in a small town in Depression-era Illinois. He notices that his students' conversations are stilted and recitation-like. 

To best help his students have more productive conversations in literature circles, Mr. Kowalski should encourage them to 

<D nominate a discussion leader who would ensure everyone participates 
O build on each others' ideas by asking follow-up questions 
CD focus on assigned roles for the literature circles 

O participate in unstructured "talk time" so that they become comfortable speaking to their peers 


Figure B1 Question#!. 
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Task 

Question Number 

MET1 

2 of 26 




Assessment Tools 



1 ► 

| Next 



Ms. Reinhardt has her sixth- and seventh-grade students read the following 
poem by Emily Dickinson. 

I’m nobody! Who are you? 

I'm nobody! Who are you? 

Are you nobody, too? 

Then there's a pair of us—don't tell! 

They'd banish us, you know. 

How dreary’ to be somebody! 

How public, like a frog 

To tell your name the livelong day 

To an admiring bog! 

To make the poem's meaning more accessible, Ms. Reinhardt wants to 
relate the poem to a topic of interest to her students. She decides to focus 
on teen celebrities, a current hot topic in her classroom. 


Which of the following is the best option to help students use their knowledge 
about teen celebrities to understand this poem? 

Ms. Reinhardt should have students think about the way teen celebrities are 

CD often on display, with the media focusing on their every move 

CD admired by the public regardless of their positive or negative 
contributions 

CD celebrated m the media for a short time but then hardly thought about 

CD sometimes immature and therefore not able to handle the pressures of 
fame 


Figure B2 Question #2. Ym Nobody by Emily Dickinson. Reprinted by permission of Dover Publications. 


Task 

Question Number 


Assessment Tools 

MET1 

3 of 26 




► 

Next 


VbTulc students arc reading independently, Ms. Kumar begins talking with Doug, a struggling fourth-grade reader, about the 
following two sentences he just read from a book about a girl and her dog. 

I have a dog named Glover. That mischievous mutt is ahvays getting into trouble. 

Doug comments that his dog is bad too, and that he is excited to read more about the two dogs in the story. Ms. Kumar is 
trying to figure out why Doug is misreading the text. 

Which of the following is the best explanation for Doug’s confusion? 

CD Doug has not comprehended the word "mischievous." 

CD Doug has made a connection to his dog, w’hich has interfered with his comprehension. 

CD Doug does not understand that "Glover" and "mischievous mutt" refer to the same dog. 

O Doug is confused by the complex sentence structures used m the text. 


Figure B3 Question #3. 
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Task 

Question Number | 


Assessment Tools | 

MET1 

4 of 26 




Questions 4-5 are based on the following teaching scenario 

Mr. Goldberg is teaching the science fiction novel The Giver, by Lois 
Lowry, to his sixth- and seventh-grade students. The people in the society 
presented in the novel have no knowledge of the world outside their 
community. Only one person in the society, known as "the Giver," holds 
knowledge of the rest of world. The main character in the novel, Jonas, is 
designated as the Giver at age 12. In the passage below, Jonas is startled by 
the look of a newborn baby's eyes. Mr. Goldberg reads the passage aloud to 
the class: 


But he had been startled by the newchild's eyes. Mirrors were rare in 
the community; they weren’t forbidden, but there was no real need of 
them, and Jonas had simply never bothered to look at himself very 
often even when he found himself in a location where a mirror existed. 
Now, seeing the newchild and its expression, he was reminded that the 
light eyes were not only a rarity but gave the one who had them a 
certain look—what was it? Depth, he decided; as if one were looking 
into the clear water of the river, down to the bottom, where things 
might lurk which hadn't been discovered yet. He felt self-conscious, 
realizing that he, too, had that look. 


Mr. Goldberg stops reading when he realizes that some students seem 
confused. 

Which of the following activities would best help Mr. Goldberg's students 
comprehend the passage? 

O Visualize Jonas and the "newchild" looking at each other. 

CD Use evidence from the text to show how Jonas was feeling. 

O Focus on Lowry's use of italics to emphasize certain words. 

O Describe a time when you felt self-conscious like Jonas. 


Figure B4 Question #4. 


Task 

Question Number 

MET1 

5 of 26 


Questions 4-5 are based on the follow ing teaching scenario. 

Mr. Goldberg is teaching the science fiction novel The Giver, by Lois 
Lowry, to his sixth- and seventh-grade students. The people in the society 
presented in the novel have no knowledge of the world outside their 
community. Only one person m the society, known as "the Giver," holds 
knowledge of the rest of world. The main character in the novel, Jonas, is 
designated as the Giver at age 12. In the passage below, Jonas is startled by 
the look of a newborn baby's eyes. Mr. Goldberg reads the passage aloud to 
the class: 


J L 


| Assessment Tools 

1 

Z> 1 

Reuit:iu| 

► 

Next 


- 

Which of the following questions w ould best focus a class discussion on the 
theme of the passage? 


CD Why is Jonas startled when he looks into the "newchild's" eyes? 
CD Why is Jonas chosen to be the Giver? 

O Why are there so few mirrors in the community? 

CD Why has Jonas never bothered to look at himself in a minor? 


But he had been startled by the new'child's eyes. Mirrors were rare in 
the community; they w'eren't forbidden, but there was no real need of 
them, and Jonas had simply never bothered to look at himself very 
often even w'hen he found himself in a location w-here a mirror existed. 
Now, seeing the newchild and its expression, he w r as reminded that the 
light eyes were not only a rarity but gave the one w'ho had them a 
certain look—what w'as it? Depth, he decided; as if one w-ere looking 
into the clear w-ater of the river, down to the bottom, w'here things 
might lurk w'hich hadn't been discovered yet. He felt self-conscious, 
realizing that he, too, had that look. 



Figure B5 Question #5. 
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Mr. Wong and his fifth-grade students are studying the impact of humans 
on the environment. The students in Mr. Wong's class are avid readers, 
often discussing and writing about fiction that they read. Because his 
students are strong readers, Mr. Wong is surprised that many of them 
struggle to comprehend informational text. He gives his students the 
following passage to read in pairs and notes that they can read it fluently. 


In the discussion that follows, many students seem to have trouble 
comprehending the information about soil. 


Save the Soil! 

Soil seems to be everywhere—in fields, in backyards, clinging to the 
soles of our shoes. But the nation's fertile soil is vanishing at an 
alarming rate. For every bushel of com produced in the U.S., about a 
bushel of soil disappears. 

How? Rain can wash loose soil into streams and rivers. To replace an 
inch of washed-awav topsoil, plants and other matter on the surface 
must break down for hundreds or even thousands of years. Soil 
disappears when houses or malls are built on land where crops could be 
planted. Pollution also ruins soil, making it unsafe for planting. 


For each text feature, indicate whether it will be likely to cause difficulty. 



Likely to cause 
difficulty 

Not likely to 
cause difficulty 

The connection between "fertile soil" 
and "bushel of com produced" 



The synonymous use of "fertile soil" 
and "topsoil" 



Starting the second paragraph with the 
one-w'ord sentence "How?" 



The lack of an explicit statement that 
humans play a role in erosion 



The use of subordinate clauses in a 
number of sentences 



The description in the first sentence of 
the text of w'here the soil might be 
found 




zJ 


Figure B6 Question #6. Adapted from TIME for Kids Magazine , 2/9/1999 © 1999 Time Inc. Used under license. TIME for Kids and 
Time Inc. are not affiliated with, and do not endorse, products or services of Educational Testing Service. 
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| Next question 


Questions 7-9 are based on a lesson using Jerry ■ Spinelli's novel "Maniac Magee ." 

Mr. Haddad is using Jerry' Spinelli's novel Maniac Magee to teach his fifth-grade class about how authors develop their 
characters. 

Which of the following questions would be the best choice to help his students focus on the essential features of character 
development? 

O ^at kinds of symbolism does the author use to develop Maniac Magee's character? 
o Can you describe where Maniac Magee lives? 

O How' does Maniac Magee respond to trouble? 

O What do you think might have happened to Maniac Magee before this story began? 


Figure B7 Question #7. 
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Figure B8 Question #8. 
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Next 



Questions 7-9 are based on a lesson using Jerry' Spinell i’s novel"Maniac Magee ." 

Mr. Haddad asks his students to create a character map—a graphic organizer in which students record words and short 
phrases that capture the most important aspects of a character's actions and memorable sayings, as well as how other 
people react to the character. Rachel has chosen John McNab, a minor character who appears infrequently in Maniac 
Magee. Mr. Haddad notices that Rachel has been copying every word in the novel about John McNab into her notebook. 

Which of the following is the best teaching approach to help Rachel record useful information about her character’s 
development? 

O Interview her about what John McNab says and does at important points in the story. 

O Suggest that she use a character map of Maniac Magee as a model for her own work. 

O Show her how f to discriminate between important and unimportant information m text. 

O Help her create a list of events that involve John McNab, and model how to write about the first event. 


Figure B9 Question #9. 
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Questions 10-11 are based on the follow ing teaching scenario. 


Mr. Foreman is teaching his sixth- and seventh-grade students a unit that 
focuses on strategies for reading informational text. His textbook includes a 
resource section with ideas that can help students identify the organization 
of text passages. The following graphic organizers are included in the 
textbook's resource section: 


Description or List 


Compare and Contrast 

GD 


J_L 


To best help his students improve their reading of informational text, Mr. 
Foreman should have them use these graphic organizers to 

O record and focus on the meaning of key vocabulary in the text 

O comprehend that nonfiction texts are used to extract factual 
information 

O identify concrete visual images to recall important details 

O anticipate patterns of information by identifying commonly used 
structures 


Assessment Tools 



Sequence/Time order 


Cause and Effect 


0-0-0 EH 


Mr. Foreman is considering how f to use these examples to help his students 
understand how- graphic organizers can support their comprehension of 
informational text. 


Figure BIO Question #10. 


Task || 

|| Question Number 

MET1 

11 of 26 



Assessment Tools 

1 

I 

_ 

► 

Hex 


Questions 10-11 are based on the follow ing teaching scenario. 

Mr. Foreman is teaching his sixth- and seventh-grade students a unit that 
focuses on strategies for reading informational text. His textbook includes a 
resource section with ideas that can help students identify- the organization 
of text passages. The following graphic organizers are included in the 
textbook's resource section: 


Description or List 


Sequence/Time order 


Compare and Contrast 

GD 


Cause and Effect 


0-0-0 O 


Mr. Foreman is considering how- to use these examples to help his students 
understand how- graphic organizers can support their comprehension of 
informational text. 


Mr. Foreman is thinking about helpful information to share with his students 
about these graphic organizers. 

For each claim, indicate whether it is an accurate statement. 



Accurate 

statement 

Not an 

accurate 

statement 

"Cause and Effect" cannot always be used 
because effects sometimes precede causes 
in an essay. 



Sometimes more than one graphic organizer 
can be used because some essays have 
multiple text structures. 



"Compare and Contrast" is the most useful 
graphic organizer for nonfiction. 



There are more nonfiction text structures 
than those presented by these graphic 
organizers. 



"Description or List" and "Sequence Time 
Order" are ahvavs chronological. 



These graphic organizers can be used 
interchangeably. 




Figure B11 Question #11. 
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Ms. Jackman has noticed that the English-language learners in her 
classroom are proficient in conversational English and skilled at reading 
fiction. However, these students have difficulty reading textbooks in science 
and social studies and generally do not comprehend those texts as well. Ms. 
Jackman's class is about to read the following text as part of a unit on 
endangered species. 

Cougars, often called mountain lions, are solitary' and secretive animals 
that are seldom seen in the wild. Cougars are known for their strength, 
agility, and ability to jump. Their powerful legs allow' them to leap 30 
feet from a standstill, or to jump 15 feet straight up a cliff walL A 
cougar’s strength and pow'erful jaw's allow' it to take dowii and drag 
prey r larger than itself. 

Cougars are the largest members of the cat family in North America. 

Adult males w’eigh between 140 and 180 pounds and measure 7-8 feet 
long from nose to tip of tail. They vary in color from reddish-browTi to 
gray, with a black tip on their long tail. 

Ms. Jackman reads the text in advance to anticipate the difficulties her 
English-language learners may encounter. 


For each text feature, indicate whether it would represent a reading challenge 
for the English-language learners in Ms. Jackman's class. 



Would represent a 
reading challenge 
for English- 
language learners 

Would not represent 
a reading challenge 
for English-language 
learners 

Text not in chronological 
order 



Words like "wild" and 
"mountain" 



Words like "seldom" and 
"often" 



Sentences without the w'ord 
"cougar," e.g., "Their 
powerful legs..." and "They 
vary in color..." 



References to units of 
measure in feet and pounds 




Figure B12 Question #12. 
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Ms. Meyer is reading her fourth-grade students' papers to identify short passages that demonstrate good use of details in 
their writing. 

Winch of the following samples of student writing would be best to share with the class as a model? 

0> When my mother gets angry she yells a lot, and then her voice is loud like a siren. 

<0 The big door w f as shaped like a rectangle, and it was made of fancy' w'ood. 

O She jumped so fast that her feet thumped the way a wild rabbit sounds hopping in a cage. 

O I went downstairs for breakfast, sat down at the table in a comfortable chair, and ate a delicious muffin. 


Figure B13 Question #13. 
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Questions 14-17 are based on the following teaching scenario 

Ms. Garcia is working with an advanced sixth-grade reading group. She is 
using the following excerpt from Their Eyes Were Watching God by Zora 
Neale Hurston as a model for a writing assignment. 


Ms. Garcia has recently been working on a number of literary techniques with 
her students. Before beginning the lesson, Ms. Garcia looks over the 
paragraph. 

Which of the following best describes the central literary technique used in this 
passage? 


So Janie began to think of Death. Death, that strange being with the 
huge square toes who lived way in the West. The great one w’ho lived 
in a straight house like a platform without sides to it, and without a 
roof. What need has Death for a cover, and what winds can blow’ 
against him? He stands in his high house that overlooks the w'orld. 
Stands w’atchful and motionless all day with his sword drawn back, 
w’aiting for the messenger to bid him come. Been standing there before 
there w’as a w’here or a w’hen or a then. She w f as liable to find a feather 
from his wings lying in her yard any day now'. 


o Simile 
O Alliteration 
CD Rhetorical question 
CD Personification 


Figure B14 Question #14. 
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Questions 14-17 are based on the following teaching scenario 

Ms. Garcia is working with an advanced sixth-grade reading group. She is 
using the following excerpt from Their Eyes Were Watching God by Zora 
Neale Hurston as a model for a waiting assignment. 

So Janie began to think of Death. Death, that strange being with the 
huge square toes w’ho lived way in the West. The great one w’ho lived 
in a straight house like a platform without sides to it, and without a 
roof. What need has Death for a cover, and w’hat winds can blow' 
against him? He stands in his high house that overlooks the w’orld. 
Stands watchful and motionless all day with his sw'ord drawn back, 
waiting for the messenger to bid him come. Been standing there before 
there was a w'here or a w’hen or a then. She w f as liable to find a feather 
from his wings lying in her yard any day now’. 


During a discussion of the passage, Ms. Garcia asks her students, "What effect 
do you think Hurston is trying to achieve in the last sentence?" 

Which of the following student responses provides the best explanation of the 
author's use of the feather in the passage? 

O "ft probably’ means the person is like a Greek god." 

CD "I think it is saying that the death guy is close by." 

CD "ft probably’ is meant as a symbol of beauty’ to help Janie stop thinking 
about death." 

O "I think she w’ants to get away or fly aw’av from home like a bird." 


Figure B15 Question #15. 
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Questions 14-17 are based on the following teaching scenario 

Ms. Garcia is working with an advanced sixth-grade reading group. She is 
using the following excerpt from Their Eyes Were Watching God by Zora 
Neale Hurston as a model for a writing assignment. 


Assessment Tools 

1 

ZD | 



Ms. Garcia asks students to write a poem that mimics Hurston's use of abstract 
nouns in the passage. 


For each teaching method, indicate whether it would be effective in helping 
students generate ideas for a poem that mimics Hurston's use of abstract nouns. 


So Janie began to think of Death. Death, that strange being with the 
huge square toes who lived way in the West. The great one w’ho lived 
in a straight house like a platform without sides to it, and without a 
roof. What need has Death for a cover, and w’hat winds can blow' 
against him? He stands in his high house that overlooks the w’orld. 
Stands watchful and motionless all day with his sword drawn back, 
w aiting for the messenger to bid him come. Been standing there before 
there was a w’here or a w’hen or a then. She w ? as liable to find a feather 
from his wings King in her yard any day now’. 



Effective 

Not 

Effective 

Help students brainstorm a list of abstract nouns other 
than "death" and write the students' suggestions on the 
board. 



Share a successful model from a similar lesson, wTitten by 
a student in the previous school year. 



Ask students to sketch a picture of w hat their abstract 
noun might look like if it were a person or creature. 



Ask students to examine the meaning of death in various 
cultures and traditions. 



Ask students to research the meaning of other major life 
events. 






Figure B16 Question #16. 
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Questions 14-17 are based on the following teaching scenario 

Ms. Garcia is working with an advanced sixth-grade reading group. She is 
using the following excerpt from Their Eyes Were Watching God by Zora 
Neale Hurston as a model for a wTiting assignment. 

So Janie began to think of Death. Death, that strange being with the 
huge square toes who lived way in the West. The great one w’ho lived 
in a straight house like a platform without sides to it, and without a 
roof. W’hat need has Death for a cover, and w’hat winds can blow’ 
against him? He stands m his high house that overlooks the w’orld. 
Stands w’atchftil and motionless all day with his sw’ord drawn back, 
w’aiting for the messenger to bid him come. Been standing there before 
there w r as a w’here or a w’hen or a then. She w’as liable to find a feather 
from his wings lying in her yard any day now’. 


Assessment Tools 

1 

| — 1 

► 

He* 


Latoya is a gifted student w’ho tends to finish her assignments before anyone 
else in the class. Her w’ork is consistently satisfactory, but Ms. Garcia senses 
that she is capable of developing her WTiting even further. Ms. Garcia reads 
Latoya's first draft. 

Fear walks around all by herself 
And no one can see her. 

She has straight brown hair 

And her blue eyes droop above her bags. 

She follow’s person after person. 

But the one she is looking for. 

She win never find... 

Happiness. 

Which of the foUowing strategies w’ould be the most effective for helping 
Latoya develop a second draft that mimics Hurston's passage? 

O Ask Latoya to elaborate by using specific details and action verbs to 
describe w’hat Fear looks like and how’ she moves. 

O Ask Latoya to list images, animals, or objects that might be used to 
characterize Fear and Happiness. 

O Ask Latoya to read her paragraph aloud and consider how’ her audience 
might understand the passage. 

O Ask Latoya to reread the excerpt from the novel and compare it to her 
draft. 


Figure B17 Question #17. 
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Ms. Verde is working with her sixth-grade students on persuasive writing. She wants her students to choose a position and 
then argue either for or against that position. She assigns the topic, "Should students be required to wear school uniforms 
at school?" Ms. Verde begins the lesson with a whole group activity in which students quickly list as many ideas as they 
can think of to support both sides of the argument. 

Explain how generating ideas for both sides of a topic could support students in writing a persuasive argument. 


Figure B18 Question #18. 



Figure B19 Question #19. 
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Assessment Tools 


Ms. Figueroa and her students are reading an excerpt from the Wallace 
Stevens poem "Anecdote of the Prince of Peacocks." 


I knew the dread 
Of the bushy plain, 

And the beauty 
Of the moonlight 
Falling there. 

Falling 
As sleep falls 
In the innocent air. 

After discussing the definition of simile, Ms. Figueroa asks students to write 
down as many similes as they can think of that describe moonlight. She 
notices that Regine, a struggling student, isn't writing at all. When Ms. 
Figueroa asks her why, Regine says, "Writing a simile about moonlight 
doesn't make sense. Moonlight is just like moonlight, it isn't like anything 
else." 


Which of the following teaching responses is most likely to help Regine write a 
simile for moonlight? 

O "Of course it isn't exactly like anything else. What you need to do is 
think about a comparison." 

O "What color is the moonlight? Can you think of something else that has 
the same color?" 

O "The sunlight is a lot like moonlight, isn't it? Except sunlight is warm 
and moonlight isn't" 

O) "Remember last week when we talked about how the sun is a flame? 
We are doing something similar here." 


Figure B20 Question #20. 
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Ms. Okeke's fifth-grade class is reading a short story entitled "The Terrible Summer." She wants to use sentences from 
this story to model how the author uses the technique of "showing" rather than "telling" in her writing. 

Which of the following sentences should Ms. Okeke select to best demonstrate the concept of "showing"? 

O The sun flamed like a giant match, and the mercury w’as hovering at a hundred degrees. 

O It was such a feverish afternoon, everyone felt like they were in an oven. 

O The asphalt bubbled, and ice cream cones melted before the fifth hck. 

O Bill detested the sweltering summer, so he conspired to stay in the delicious coolness of his basement. 


Figure B21 Question #21. 


ETS Research Report No. RR-14-33. © 2014 Educational Testing Service 


43 




































G. Phelps etal. 


Developing Content Knowledge for Teaching Assessments 


Figure B22 Question #22. (Question #22 in this assessment cannot be shown due to copyright restrictions for an excerpt used from a 
published text in this question.) 
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Mr. Jackson's sixth- and seventh-grade students are writing flash fiction. After reading the following draft of Arielle's short story, 
Mr. Jackson wants to point out some of the strengths in her writing. 

Natasha left the house and walked sadly down the sidewalk, slowly shuffling her feet. As she walked by the old house, she 
was surprised by how sad she felt. She stepped onto the grass, and the sprinklers sprayed her face. The sprinkle of cold water 
caught her by surprise. Her brother shouted down the street asking her to wait up. Thankfully he couldn't tell the tears from 
the drops of water. 

For each writing feature, indicate whether it describes a strength in Arielle's writing. 



Describes a strength 
in Arielle's writing 

Does not describe a strength 
in Arielle's writing 

Strong character development 



Use of figurative language 



Conveying emotions through showing 



Use of powerful images 




Figure B23 Question #23. 
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Mr. Abie's sixth-grade students have just begun writing personal narratives. Manuel produces the 
following first draft. 

Cars are cool. I collect them. They can open the doors & trunk. I can make a list of the cars I 
know 1st is a Ferari f50, a Viper GTs, orrr a Mustang GT, Lamberginy, corvette. Lexis GT, 1800 
Honda GT, Porshe 911 Torbo, Mercedes, Mini. The Mini is cool. It is small and has mini spinners. 
I imagine where I could ride in my car. I drive in my Mini all day and go to places MC Donald's 
any time I want. I love car's there just really special to me. There really cool that's wy I like them. 
If I'm rich I would buy lots of cars. My favorite car is my hmo it light's up it's really cool. It 
reminds me of mv to Disnev land. 


Mr. Able and Manuel have the following conversation during a WTiting conference. 

Mr Able: It seems like you’re really interested in collecting cars, that you're really passionate about 

it. How* did you get started? 

Manuel I collect cars because they're cool. I was going to bring my cars in for Expert Day. 

Mr Able: Let's slow' down for a minute. What w r as the first car you ever collected? 

Manuel My mom saw' a show- on Disney channel about kids w'ho collect things. 

Mr. Able: Can you tell me about your first car? Do you remember getting it? 

Manuel : Um... it was my yellow' Corvette. 

Mr Able: And how' did you get your yellow' Corvette? Was it a present? 


In this WTiting conference, Mr. Able is attempting to 
help Manuel improve his narrative by 

O focusing his story on a single idea 
O developing more vivid descriptions 
O including a broader perspective on his topic 
O explaining why he likes collecting things 


Figure B24 Question #24. 
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Mr. Blunk's fourth-grade students are learning to w-rite five-paragraph research reports about how' Sioux Native Americans 
lived before the colonial era. Bruce submits the following first draft of an introductory paragraph based on his research. 

The Sioux peoples w'ere very' different from us. To live, they needed food, clothing, and shelter. All of their food, 
clothing, and shelter w r as different from ours. The Sioux w'ere extremely' interesting. In my opinion, everyone should 
be interested in the Sioux. 

To help Bruce develop his first paragraph of the research report, w'hich of the following questions w ould be best for Mr. 
Bhink to ask during a one-on-one conference? 

O "What is another w’ord you could use instead of 'interesting?'" 

"How could you vary your sentence structure to improve the flow'?" 

O "How w ould you convince someone to agree with your opinion?" 

O "What are you going to tell us about their ’food, clothing, and shelter?’" 


Figure B25 Question #25. 
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Ms. Lee is teaching a unit on persuasion to her sixth- and seventh-grade students. Although the students have read articles 
on the pros and cons of their topic—eliminating vending machines from school cafeterias—she notices that they have not 
used textual evidence in their essays. 

Which of the following is the most effective activity to help Ms. Lee's students add textual evidence to their essays to 
support their arguments? 

Ms. Lee should ask her students to 

o critically read articles about the pros and cons of having vending machines in schools 

O record evidence they find in articles on sticky notes and place the sticky notes in the appropriate places in their 
drafts 

O peer edit each others' drafts with a focus on identifying both strengths and weaknesses in the essays 
O use their writing guides to apply the proper citation style for authoritative sources 


Figure B26 Question #26. 


Appendix C 

Mathematics 6-8 Assessment 


Table Cl Item/Question Sequence Number for Mathematics 6-8 Assessment 


Item/question sequence number 

Answer key 

Item/question sequence number 

Answer key 

1 

C 

lie 

Provides 

2 

C 

Ilf 

Does not provide 

3a 

Provides 

12 

B 

3b 

Provides 

13 

A 

3c 

Does not provide 

14a 

Provides 

4 

B 

14b 

Does not provide 

5 

B 

14c 

Provides 

6a 

Provides 

14d 

Provides 

6b 

Does not provide 

15 

B 

6c 

Provides 

16a 

Provides 

6d 

Provides 

16b 

Provides 

6e 

Provides 

16c 

Provides 

7 

A 

16d 

Does not provide 

8a 

Valid 

16e 

Does not provide 

8b 

Valid 

17 

A 

8c 

Not valid 

18 

B 

8d 

Valid 

19a 

Does not demonstrate 

8e 

Not valid 

19b 

Demonstrates 

9 

D 

19c 

Demonstrates 

10 

B 

19d 

Demonstrates 

11a 

Provides 

20 

B 

lib 

Provides 

21 

C 

11c 

Does not provide 

22 

Open-ended response 

lid 

Provides 

23 

Open-ended response 
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Ms. Hupman is teaching an introductory lesson on exponents. She wants to give her students a quick problem at the end of 
class to check their proficiency in evaluating simple exponential expressions. Of the following expressions, which would be 
least useful in assessing student proficiency in evaluating simple exponential expressions? 

3 3 
2 3 
2 2 

All of these are equally useful in assessing student proficiency in evaluating simple exponential expressions. 


o 

o 

o 

o 


Figure Cl Question #1. 


[Task] 

|| Question Number 


| Assessment Tools 

MET 

2 Of 23 

mM 

i ► 
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In a unit introducing proportions, Mr. Hayes gave his students the following problem. 

Sabrina's Flow-er Shop has 24 roses and 36 carnations. What is the ratio of the number of roses to the total 
number of roses and carnations at Sabrina's Flower Shop? 


When the students finished, he w’anted to give them a similar practice problem. Of the following, w’hich asks students to do 
the most similar mathematical w-ork and thinking as the problem above? 

CD A spinner has 10 equal sections numbered 1 through 10. What are the odds that the spinner will land on a prime 
number w-hen it is spun? 

O Nick's Team Sports Store has 24 basketballs and 60 footballs on display. What is the ratio of the number of 
basketballs to the number of footballs on display? 

O For breakfast during the month of April, Katherine ate cereal on 18 days and eggs on the other 12 days. What i s the 
ratio of the number of days Katherine ate cereal for breakfast to the number of days in April? 

O All of these problems ask students to do the same mathematical w r ork and thinking. 


Figure C2 Question #2. 
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During an introductory lesson on solving equations, Mr. Chen gave his students the following problem. 

10-Q= 12-7 


He asked his students to decide what number can be put into the box to make the sentence true. Students got the correct 
answer of 5, but they used different strategies. For each of the following strategies, indicate whether or not it provides 
evidence of mathematically valid reasoning about this problem. 



Provides Evidence of 
Mathematically Valid 
Reasoning 

Does Not Provide Evidence of 
Mathematically Valid 
Reasoning 

12 - 7 - 5, so I need to find out w’hat I take away from 10 to get 5. 10 - 5 - 5, so the 
answer is 5. 



10 is two less than the 12 on the other side, so I need to take away two from 7 to get the 
same answer of 5. 



When I subtract 12 minus 7 the answer I get is 5, so 5 goes into the box to make the 
sentence true. 




Figure C3 Question #3. 
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At the start of a lesson on finding the side length of a square given its area, Ms. Sabatine gave her students a problem to 
assess their prior knowledge. Several students incorrectly answered that the side length of a square with area 36 square 
units is 9 units. At the end of the lesson, Ms. Sabatine wanted to give a similar problem to assess what her students had 
learned. Of the following areas to use in this problem, which would be least useful for assessing student learning in this 
situation? 

O 9 square units 
O square units 
O 100 square units 

O AD of these are equaDv useful for assessing student learning in this situation. 


Figure C4 Question #4. 


48 


ETS Research Report No. RR-14-33. © 2014 Educational Testing Service 





























G. Phelps etal. 


Developing Content Knowledge for Teaching Assessments 


Task 

Question Number | 


Assessment Tools 

MET 

5 0(23 


< 1 

Back | 

1 

► 

Hex’ 


To assess her students' prior knowledge about evaluating arithmetic expressions, Ms. Santiago assigned a worksheet of 
problems. She noticed that Alexis answered the first two incorrectly and the next two correctly. 

1 ) 7 x 2 - 6 + 3 - S' 

2) 9 — 5 + (16 -r 8) s £ 

3) 9 + 24-5-3-1 /(, 

4) 17-(3+7X2) =- O 

Which of the remaining problems is Alexis likely to answer incorrectly? 

O 8 + 7-12 + 3 
O 13-3x2 + 5 
O (27 + 3 - 4) + 8 
O (16-12) x 5 +10 


Figure C5 Question #5. 
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In a unit on proportional reasoning, Ms. Richmond's class was discussing the following problem. 

If 4 cups of cocoa and 2 cups of sugar yield 16 brownies, how many cups of cocoa and how many cups of sugar are needed to make 
24 brownies? 

Ms. Richmond's students used different strategies to solve the problem. Of the following, w-hich provides evidence of mathematically valid 
student thinking 0 For each strategy, indicate whether or not it provides evidence of mathematically valid student thinking. 



Provides 

Evidence of 
Mathematically Valid 
Student Thinking 

Does Not Provide 

Evidence of 
Mathematically Valid 
Student Thinking 

48 brownies need 12 cups of cocoa and 6 cups of sugar. To make 24 brownies, I need 6 cups of cocoa 
and 3 cups of sugar. 



4 and 2 both go into 16. 4 phis 2 is 6, half of 6 is 3, and 6 and 3 both go into 24, so you need 6 cups of 
cocoa and 3 cups of sugar to make 24 brownies. 



1 brownie needs j cup of cocoa and ^ cup of sugar. To make 24 brownies, I need to multiply by 24 for 
cocoa and sugar. Thus, I need 6 cups of cocoa and 3 cups of sugar. 



6 cups of cocoa and sugar makes 16 brownies, so 24 brownies need 9 cups of cocoa and sugar. Since the 
ratio of cocoa to sugar is 2:1,1 need 6 cups of cocoa and 3 cups of sugar. 



Since 1 cup of sugar is needed to make 8 brownies, I need 3 cups of sugar to make 24 brownies. The 
amount of cocoa is two times the amount of sugar in the recipe, so I need 6 cups of cocoa. 




Figure C6 Question #6. 
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Mr. Philbey gave his students the following fraction multiplication problem to evaluate. 

iiy36yllvIy_LyI 
8 5 3 612 7" 

Several of his students agreed that the answer is ^, but their approaches to the problem were very different. Of the 
following student strategies, which would noi be valid for solving this problem? 

O You can cancel the 5s and the 36s and the 2s, then you get 1 x 14 x 1 x 5 over 8x1x7, which is 70 over 56, 

10 5 

which simplifies to — or just —. 


1/ U 14 1 5 l 114x1x5 

-jr x 4 « —------ 

8 f 1 \% 1 8x1x7 


70 

56 


10 

8 


<3 You can factor and just get 5 divided by 4. 

15 36 14 1 5 2 / x $ x / x / x % x % x 1 x 7 x 5 x l 5 

S ' 5 ' 3 ' 6 U X 7‘/x/xZx/x/x;'x}r*2x2x/x7'-t 


o You can cancel the 15 and the 5 and 3, and then cancel the 6 and 12 with the 36 and the 2 from the 14. Then you 
can cancel the 7s, and you get 10 over 8, or 

O All of these are valid strategies for evaluating this expression. 


Figure C7 Question #7. 
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During a lesson on solving multistep equations, Mr. Steinbrecher asked his students to solve the equation 4(5x - 11) * 16. 
While walking around the class looking at what the students were writing, he noticed several different strategies. For each 
of the following student solutions, indicate whether or not it is a valid strategy for solving this problem. 



Strategy Is Valid 

Strategy- Is Not Valid 

£^(Sx- ll) 

Tx-ll z f 

Jr 

dU) 



V* y(s-x-ll) * He-1 

H.I SV-«) * Hr* 

yi 

Si-n * v 

til 

S*» /S" 

CE 



- >L 

fx -/* «* 

fit +H . 

r? 

cud 



H(rx-ll) * It 
£Sg~jM - !<• 

2o .»© 

*• 

CHJ> 



<4(^x-n)-- It, 

H y * * 

7» *f 

j? ' y( " s', 

<2S> 




Figure C8 Question #8. 
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Ms. Lowe is teaching a unit on statistics. Some of the students in her class speak a language other than English at home 
and when they work in small groups. As she prepares for the unit, she makes a note of vocabulary words that she believes 
will be challenging for her students, especially words that have different meanings in different contexts. Of the following 
vocabulary words that she identified, which will require the least clarification regarding differences in meaning 0 

O Mean 

O Mode 

O Range 

O Quartile 


Figure C9 Question #9. 
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During a unit on proportional reasoning, Ms. Tran gave her students the following problem. 


Two jugs each have the same amount of fruit punch. Jug #1 has 5 ounces of fruit punch concentrate for every 9 ounces of water. Jug #2 has 11 ounces 
of fruit punch concentrate for every 14 ounces of water. Which jug contains the fruit punch with the stronger taste, or will the fruit punch in both jugs 
taste the same? 


Students got the correct answer that Jug #2 contains the fruit punch with the stronger taste, but they thought about the problem in different ways. For each of 
the following student explanations, indicate whether or not it provides evidence of mathematically valid reasoning about this problem. 



Provides Evidence of 
Mathematically Valid 
Reasoning 

Does Not Provide Evidence of 
Mathematically Valid 
Reasoning 

Jug #1 has 1 j ounces of w ater for each ounce of concentrate. Jug #2 only has 1 -jj ounces 
of w’ater for each ounce of concentrate, so Jug #2 is stronger. 



11 times the amounts in Jug #1 gives 55 ounces of concentrate and 99 ounces of w’ater. 5 
times the amounts in Jug #2 gives 55 ounces of concentrate and 70 ounces of water. When 
the concentrate is the same, there is more water in Jug #1, so Jug #2 is stronger. 



—j = 1^, and the ratio of concentrate to water in Jug #1 is only 5 parts water to 9 parts 
concentrate, so Jug #2 is stronger. 



I looked at the ratios of concentrate to w’ater For Jug #1, ^ ^ - y, and for Jug #2, 

II 11 10 2 . _ 

> -jj > jz - j, so Jug *2 is stronger. 



In Jug #1, the concentrate is less than 60% of the water. In Jug =2, the concentrate is more 
than 75% of the w’ater, so Jug #2 is stronger. 



If you add 6 to both of the numbers for Jug #1, you get 11 ounces of concentrate and 15 
ounces of w’ater. This is more water than w’hat is in Jug #2 for the same amount of 
concentrate, so Jug #2 is stronger. 




Figure Cl 1 Question #11. 



Figure C12 Question #12. 
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Ms. Malloy’s textbook suggests using white chips (for positive integers) and red chips (for negative integers) to teach 
integer subtraction. To solve the problem 2 - (-3), the book describes the process below. 

1. Start with two white chips. 

2. Add three pairs of red and white chips. 

3. Then take away three red chips for the -3. 

4. There are 5 white chips left, so the answer is 5. 

Ms. Malloy is concerned that her students will see that adding red and white chips sets up the subtraction, but not 
understand why the additional chips can be added. Of the following, which provides a mathematical explanation for why 
the three pairs of red and white chips can be added? 

O By adding both a red chip and a white chip to an expression, you are really adding a zero, which does not change 
the value of the original amount. 

O The 2 is two white chips and the -3 is three red chips. When you add the three red chips, you also add three w'hite 
chips. That way, when you take aw'ay the red chips, the total number of white chips will be 5, which is the correct 
answer. 

O The three red chips are for the -3 and the three white chips are what the red chips become because the negative of 
a negative is positive. 

O Red chips represent negative integers, but when you subtract negative three, you get positive three, so you need to 
add three white chips as well as three red chips. 


Figure C 13 Question #13. 
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Ms. Spiess’s class had just finished a unit on operations with fractions. Before moving on to the next unit, she wanted to 
see if her students understood the invert-and-multipty rule for dividing fractions. 

o_ c_ a d ad 
b~d m b*c m bc 


Ms. Spiess asked her students to explain whv the algorithm works. For each of the following explanations, indicate 
whether or not it provides evidence that the student understands whv the algorithm w'orks. 



Provides 

Evidence 

Does Not Provide 
Evidence 

It’s because you can think of it as a fraction and just multiply by one to get nd of the denominator. 

a a d 

b 1 ~c a d 

- = - X - = — X — 

£ £ d b c 

d d c 



Dividing is like cross-multiplying, so you need to multiply the numerator of the first with the denominator of the second 

and then the denominator of the first with the numerator of the second. That gives you 



When vou find a common denominator vou get 7 - ■+■ ~r - — -*■ . but if vou w’ant to know r how' manv go 

b a oa od ' od 

ad ad 

into — that’s just the same as how r manv be go into ad. So, it’s -r—. 
od DC 



ad c a c a c ad 

I can see that — x — * —, so when you divide by -j you get — . 




] 


Figure C14 Question #14. 
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Ms. Jones's class has been studying area and perimeter. Some of the students in her class are learning English; they speak 
Spanish at home and when they work in small groups. One day they work on the following problem. 

• Look for all of the rectangles with area 36 (w'here length and width are integers). Write down the dimensions. 

• Calculate the perimeter for each rectangle. 

• Describe a pattern relating the perimeter and the dimensions. 

Of the following student responses, which most accurately describes the relationship between the dimensions of a 
rectangle and its perimeter? 

O A rectangle has ... two ... short sides, and two long sides. The perimeter of the ... long ... skinny rectangle is big. 
It’s about twice longer than the square. 

O The longer the ah ... the longer [traces the shape of a long rectangle with her hands several times] the ah ... the 
longer the, thing, you know' the more the perimeter, the higher the perimeter is. 

o The rectangle ... the one with 4 width and 9 length ... its perimeter 26, and its area is bigger, it’s 36. The rectangle 
3 width and 12 length ... it’s perimeter 30. It’s 4 more ... it goes up by 4 ... higher. 

O Each of these student responses describes the relationship with equal accuracy. 


Figure Cl5 Question #15. 
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Ms. Kamp asks her students to consider squares of different side lengths with only the boxes along the sides shaded as in the figure 
on the left. 

She asks each student to wTite an expression for the number of shaded boxes in a square with a side length of it boxes and to 
explain why the expression gives the number of shaded boxes for any size square. For each of the following explanations, indicate 
whether or not it provides evidence that the student understands whv the expression can be used to find the number of such shaded 



Provides 

Evidence 

Does Not Provide 
Evidence 

If you start on the bottom left and go to just below the top left, then start with the top left and go to just before the top 
right, and keep doing that, you win get 4 groups, and each group has 1 less than it, so you get 4(n - 1). 



My expression is it* 2(n - 1) + (it - 2) because the top of the square has it shaded boxes, then each of the sides has 
it - 1 shaded boxes left, and then the bottom has it - 2 shaded boxes left. 



Inside the square with a side length of it boxes is a square with side length of it - 2 boxes, so if you find the area of the 
two squares and subtract them, you will find the number of shaded boxes. So, I get it 2 - (it - 2) 2 . 



I get 4 (it - 2) + 4 because there are 36 boxes shaded, and when you put 10 in for the it in 4(it - 2) + 4 and follow' the 
order of operations, the answ'er is 36. 



My expression is 2it + 2(it - 2) because if you simplify' 2it + 2(it - 2) you get 2it * 2it - 4, w'hich is equal to 4 it - 4, 
and because this doesn't depend on it, it works for any it. 




Figure C16 Question #16. 
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Ms. Jensen had been working on rational numbers with her students for some time, and she wanted them to understand 
that between any two rational numbers, one can always find another rational number. To support this thinking, she asked 

her students to first think about how to find a fraction between any two positive fractions, such as j and \. As the class 
worked on the problem, one student said, "I think you can find a fraction between any two positive fractions by adding the 
denominators and numerators—here you just get j —- j, which is greater than j and less than \." 

Of the following descriptions, w'hich best characterizes the student's technique for finding a number between any two 
positive fractions? 

O The technique is a simple method that w'orks for any two positive fractions. 

O The technique only works for some positive fractions. 

O The technique produces messy answ’ers in many cases and should be avoided. 

O The technique should have been to find the average of the two fractions, i.e., + = + = 


Figure C17 Question #17. 
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Ms. Delgado's students are reasoning about whether or not a given number is divisible bv 3. She is concerned that even 
though her students do w’ell listing multiples of numbers, they have not established the general property that if a number is a 
multiple, then w f hen it is written as a sum of two numbers, both must be multiples or neither is a multiple. She recognizes 
that additional work is needed for the idea to be meaningful to the whole class. 

Of the following student explanations for why 80 is not divisible by 3, which uses this general property? 

O if I have one more, it is a multiple of 9. So, 80 is not divisible by 3. 

O Every 10 leaves 1 remainder, so 80 has 8 remainders. You can't divide 8 into 3 equal numbers, so 80 is not divisible 
by 3. 

O First, 80 is divisible by 2. If I divide by 2 again and again, I got 40, 20, 10, and 5.1 can't find 3 anywhere, so it is 
not divisible by 3. 

O F^ch of these explanations uses this general property. 


Figure C18 Question #18. 
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Mr. Whalen wants to review strategies for finding the greatest common factor of two numbers with his students. At the 
beginning of class, he asks his students to find the greatest common factor of 180 and 84 and to explain how they found it. 
While walking around the class looking at what the students are writing, he notices several different strategies. For each of 
the following responses, indicate w'hether or not it demonstrates a valid strategy for finding the greatest common factor of 
any two numbers. 



Figure C19 Question #19. 
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Ms. Lang's class had been studying the concept of slopes of lines, so she asked them to consider all of the lines passing 
through one point and how the slopes of those lines vary. The students had used geoboards in some earlier work, so they 
started talking about the slopes of lines on an "infinitely extended" geoboard. (Geoboards are flat blocks of wood, roughly 
one foot square, with pegs laid out on a grid w'here rubber bands can be hooked to make lines or polygons.) The students 
decided that the pegs of the infinite geoboard could be thought of as the set of points with integer coordinates in the 
Cartesian plane. 

During the discussion, students had the following exchange. 

Yonah: On the geoboard, you can't get all of the slopes, because the geoboard points are too spread out—there are a 
w’hole bunch of lines between the ones you can make. 

Andy’: I disagree. I think w r e can make any slope. Starting at one point, by choosing another geoboard point far- 

enough away, w’e can tilt the line as much or as little as w’e like. 

Becky': What I w'as thinking w'as if you run a line through one geoboard point, it win always hit another one far 
enough out. 

Of the following concepts, w’hich is most directly' related to the mathematics underlying this discussion? 

O Interpretation of the derivative—the derivative is the slope of the tangent line. 

O Density of numbers on the real line—the rational numbers are dense, but not every' real number is rational. 

O The parallel postulate—given a point and a line, there is a unique line through the given point parallel to the given 
line. 

O Each of these concepts is equally' related to the underlying mathematics. 


Figure C20 Question #20. 
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Mr. Evans asked his students to compare yj t0 yg 211(1 cleci< k which ls greater. During the class discussion, Wanda explained her idea. 

It’s like if our class had 13 boys and 15 girls and then we got 2 more boys and 3 more girls. But y is less than jy . If you add 2 to the 
top and 3 to the bottom it's yy , so then y| has to be ^ ess than y| • 


Of the following, w’hich best characterizes Wanda's reasoning? 

O Her reasoning is incorrect, because you can't just add numerators and add denominators like she did. 

O Her reasoning is incorrect, because she needs to compare and directly. 

lj lo 

O Her reasoning is correct, but it depends on it being easy to compare the ratio of the differences to one of the other ratios. 

Her reasoning is correct, but it does not w’ork in general—for instance, it wouldn't w'ork if the ratio of the added boys to the added girls were greater 

O 13 9 

than —, such as —. 


Figure C21 Question #21. 
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Mr. Yusuf is teaching a lesson on extending patterns. He plans to show his students the two patterns below m which each 
figure is constructed with toothpicks. Each pattern shows the first three figures in the pattern. He will ask his students to 
write an algebraic expression to represent the number of toothpicks in the nth figure of each pattern. 



Pattern 2: Q |~~T~| l~l~l~l 

Mr. Yusuf wants to begin with the pattern that is mathematically easier and then move to the pattern that is more complex. 
Which pattern should Mr. Yusuf give to his students first? 

O Pattern 1 
O Pattern 2 

Explain why the pattern you chose is the mathematically easier pattern of the two patterns given. 


Figure C22 Question #22. 
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A lesson in Ms. Hagerman's textbook defines the distributive property, but the exercises merely ask for its definition. To 
motivate her students to learn the definition, Ms. Hagerman wants to give an example that will illustrate the usefulness of 
the distributive property. She notices the following four examples in the teacher edition of the textbook. 


>• 12x (H 


2 - H?-*i 




4 .3«*(n + ?i 


Ms. Hagerman decides that example number 4 w ould be the best for her to use of the four examples. 

Explain w f hv example number 4 is the best example for Ms. Hagerman to use for illustrating the usefulness of the distributive property. 


Explain why the other examples do no! illustrate the usefulness of the distributive property as w’ell as example number 4 docs. 


Figure C23 Question #23. 
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English Language Arts (ELA) 7-9 Assessment 


Table D1 Item/Question Sequence Number for English Language Arts (ELA) 7-9 Assessment 


Item/question sequence number 

Answer key 

Item/question sequence number 

Answer key 

1 

A 

15 

Lea, Akeem, Sarah 

2 

A 

16a 

Literal 

3a 

Not likely 

16b 

Interpretive 

3b 

Not likely 

16c 

Personal 

3c 

Likely 

17 

A 

3d 

Not likely 

18 

Open-ended response 

4 

B 

19a 

Will 

5 

A 

19b 

Will 

6 

B 

19c 

Will not 

7a 

Should 

19d 

Will not 

7b 

Should not 

20 

B 

7c 

Should 

21a 

Has 

7d 

Should not 

21b 

Has not 

8 

D 

21c 

Has 

9 

C 

22 

D 

10 

D 

23a 

Does not describe 

11a 

Accurate 

23b 

Does not describe 

lib 

Accurate 

23c 

Describes 

11c 

Not accurate 

23d 

Describes 

lid 

Accurate 

24a 

Likely 

lie 

Not accurate 

24b 

Not likely 

Ilf 

Not accurate 

24c 

Likely 

12 

Open-ended response 

24d 

Likely 

13 

D 

25 

A 

14 

B 

26 

B 
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Mr. Hernandez is considering ways to teach his students to identity text structures prior to reading informational text. 
Which of the following would be the best activity for this purpose? 

O Scanning the text for the types of transition words used in the text passage 
O Previewing the length of the text passage to see how many subtitles it contains 
O Scanning the pictures and captions that accompany the text passage 
CD Making a K.-W-L chart to activate pnor knowledge on the topic 


Figure D1 Question#!. 
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Ms. Reinhardt has her sixth- and seventh-grade students read the following 
poem by Emily Dickinson. 

I'm nobody! Who are you? 

I'm nobody! Who are you? 

Are you nobody, too? 

Then there's a pair of us—don't teU! 

They'd banish us, you know. 

How dreary to be somebody! 

How public, like a frog 

To teU your name the livelong day 

To an admiring bog! 

To make the poem's meaning more accessible, Ms. Reinhardt wants to 
relate the poem to a topic of interest to her students. She decides to focus 
on teen celebrities, a current hot topic in her classroom. 


Assessment Tools 


Which of the following is the best option to help students use their knowledge 
about teen celebrities to understand this poem? 

Ms. Reinhardt should have students think about the way teen celebrities are 

O often on display, with the media focusing on their every move 

O admired by the public regardless of their positive or negative 
contributions 

O celebrated in the media for a short time but then hardly thought about 

O sometimes immature and therefore not able to handle the pressures of 
fame 


► 

Next 


zi 


Figure D2 Question #2. I’m Nobody by Emily Dickinson. Reprinted by permission of Dover Publications. 


Task 

Question Number 


Assessment Tools 

MET1 

3 of 26 


1 

1 

► 

Next 


Mr. Allen is working with a class of English-language learners. Although his students have become more proficient at 
conversing in English, they still often have trouble comprehending wiitten text. He is considering possible reasons that 
might explain why his students could have difficulty comprehending the following two sentences. 

Yesterday I went to the opera. The soprano’s singing w f as superb. 

For each possible reason, indicate whether it is a likely source of confusion for Mr. Allen's English-language learners. 



Likely source 
of confusion 

Not a likely source 
of confusion 

Difficulty' understanding the word "singing" 



Lack of firsthand experience with opera 



Not knowing that "soprano" is an element of the opera 



Misunderstanding the use of alliteration 




Figure D3 Question #3. 
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Questions 4-5 are based on the following teaching scenario 

Mr. Goldberg is teaching the science fiction novel The Giver, by Lois 
Lowry, to his sixth- and seventh-grade students. The people in the society 
presented in the novel have no knowledge of the world outside their 
community. Only one person in the society, known as "the Giver," holds 
knowledge of the rest of world. The main character in the novel, Jonas, is 
designated as the Giver at age 12. In the passage below, Jonas is startled by 
the look of a newborn baby's eyes. Mr. Goldberg reads the passage aloud to 
the class: 

But he had been startled by the newchild's eyes. Mirrors were rare in 
the community; they weren't forbidden, but there was no real need of 
them, and Jonas had simply never bothered to look at himself very 
often even when he found himself in a location where a mirror existed. 
Now, seeing the newchild and its expression, he was reminded that the 
light eyes were not only a rarity but gave the one who had them a 
certain look—what was it? Depth, he decided; as if one were looking 
into the clear water of the river, down to the bottom, where things 
might hirk which hadn't been discovered yet. He felt self-conscious, 
realizing that he, too, had that look. 


Mr. Goldberg stops reading when he realizes that some students seem 
confused. 

Which of the following activities would best help Mr. Goldberg’s students 
comprehend the passage? 

O Visualize j 0 nas and the "newchild" looking at each other. 

O Use evidence from the text to show how Jonas was feeling. 

O Focus on Lowry's use of italics to emphasize certain words. 

O Describe a time when you felt self-conscious like Jonas. 


Figure D4 Question #4. 


Task | 

| Question Number 
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Questions 4-5 are based on the following teaching scenario. 

Mr. Goldberg is teaching the science fiction novel The Giver, by Lois 
Lowry, to his sixth- and seventh-grade students. The people in the society 
presented in the novel have no knowledge of the world outside their 
community. Only one person in the society, known as "the Giver," holds 
knowledge of the rest of world. The main character in the novel Jonas, is 
designated as the Giver at age 12. In the passage below, Jonas is startled by 
the look of a newborn baby’s eyes. Mr. Goldberg reads the passage aloud to 
the class. 


Assessment Tools 



Which of the following questions would best focus a class discussion on the 
theme of the passage? 

O Why is Jonas startled when he looks into the "newchild's" eyes? 

O Why is Jonas chosen to be the Giver 0 
O Why are there so few mirrors in the community? 

O Why has Jonas never bothered to look at himself in a mirror? 


But he had been startled by the newchild's eyes. Mirrors were rare in 
the community; they weren't forbidden, but there was no real need of 
them, and Jonas had simply never bothered to look at himself very 
often even when he found himself in a location where a mirror existed. 
Now, seeing the newchild and its expression, he was reminded that the 
light eyes were not only a rarity but gave the one who had them a 
certain look—what was it? Depth . he decided; as if one were looking 
mto the clear water of the river, down to the bottom, where things 
might lurk which hadn't been discovered yet. He felt self-conscious, 
realizing that he, too, had that look. 


Figure D5 Question #5. 
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As part of a unit on Romeo and Juliet, Ms. Bergan wants to work with her students to develop their understanding of plot structure. The curriculum materials 
Ms. Bergan is using suggest that she introduce the following plot diagram from a textbook to help her students understand the plot structure of Romeo and 
Juliet. 



When she introduces this diagram, her students share different ideas about w-hether it does in fact represent the plot of Romeo and Juliet. 

Antonio: I think that the climax happens in the middle when Tybalt dies because after that everything is downhill. 

Sally. Isn't the climax of Romeo and Juliet at the very end w'hen they die and shouldn't the resolution pan of the diagram be a lot shorter? 

Lin. I think the climax is the fight at the beginning and the rest of the story is all a big long resolution of that feud. 

Tiffany' I think the climax comes near the beginning with the wedding because it is a love story and that is the most important part of the play. 

Ms. Bergan decides to represent students' ideas on the chalkboard in order to begin a discussion of the various interpretations of the climax of Romeo and 
Juliet. 

Which of these is an accurate representation of the students’ ideas about the climax of Romeo and Juliefl 





Figure D6 Question #6. 
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Assessment Tools 

1 

^ 1 

► 

Next 


Questions 7-9 are based on the following teaching scenario. 

Mr. Liu is teaching Harper Lee's To Kill a Mockingbird to his eighth-grade 
English class. The novel is set in a small town in Alabama in the 1930s and 
is narrated by Scout, an outspoken young girl who is the daughter of 
attorney Atticus Finch. A major conflict is introduced when Atticus defends 
a black man, Tom Robinson, who has been wrongly accused of assaulting a 
white woman. As Atticus works to prove Tom's innocence, Scout and her 
brother Jem experience harassment at the hands of various townspeople. 


Mr. Liu is aware of the difficulties his students are likely to encounter while 
trying to comprehend these aspects of the story. He is considering topics to 
include in his instruction in order to help his students better understand the 
conflict in the story. 

For each topic, indicate whether Mr. Liu should include it in his instruction. 



Should 

include 

Should not 

include 

Why a black man might not have been 
adequately represented in the legal system of the 
time 



How southern authors like Harper Lee help us to 
understand our history 



How r social attitudes are reinforced through 
teasing and bullying 



How, even today, people speak badly of lawyers 




Figure D7 Question #7. 
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Questions 7-9 are based on the following teaching scenario 

Mr. Liu is teaching Harper Lee's To Kill a Mockingbird to his eighth-grade 
English class. The novel is set in a small town in Alabama in the 1930s and 
is narrated by Scout, an outspoken young girl who is the daughter of 
attorney Atticus Finch. A major conflict is introduced when Atticus defends 
a black man, Tom Robinson, who has been wrongly accused of assaulting a 
white woman. As Atticus works to prove Tom's innocence. Scout and her 
brother Jem experience harassment at the hands of various townspeople 


Mr. Liu realizes that his students are focusing on nine-year-old Scout as just 
another character in the novel. He wants his students to focus on Scout as the 
narrator. 

Which of the following questions will best help Mr. Liu's students focus on 
Scout as the narrator of the novel? 

CD How is Scout’s behavior unusual for a girl of her social standing in the 
1930s South? 

CD How does Scout's relationship with Atticus develop during the novel? 
CD How does Scout change over the course of the novel? 

CD How does Scout’s age influence what she sees and w'hat adults tell her 
about these events? 


Figure D8 Question #8. 
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Questions 7-9 are based on the following teaching scenario 

Mr. Liu is teaching Harper Lee's To Kill a Mockingbird to his eighth-grade 
English class. The novel is set in a small town in Alabama in the 1930s and 
is narrated by Scout, an outspoken young girl who is the daughter of 
attorney Atticus Finch. A major conflict is introduced when Atticus defends 
a black man, Tom Robinson, who has been wrongly accused of assaulting a 
white woman. As Atticus works to prove Tom's innocence, Scout and her 
brother Jem experience harassment at the hands of various townspeople. 



Mr. Liu is preparing a lesson that includes the following passage from the 
middle of the novel 


Presently I picked up a comb from Jem's dresser and ran its teeth along the 
edge. 

"Stop that noise," Atticus said. 

His curtness stung me. The comb was midway in its journey, and I banged 
it down. For no reason I felt myself beginning to cry, but I could not stop. 
This w f as not my father. My father never thought these thoughts. My 
father never spoke so. Aunt Alexandra had put him up to this, somehow'. 
Through my tears I saw Jem standing in a similar pool of isolation, his 
head cocked to one side. 


Which of the following is the most likely explanation for w f hy an English- 
language learner might struggle to comprehend this passage? 

O Confusion over the sequence of events in the passage 

<3 Difficulty understanding w'hy Scout began to cry for "no reason" 

O Confusion over the sentence "His curtness stung me" 

O Difficulty recognizing that "Atticus" and "my father" refer to the same 
person 


Figure D9 Question #9. 
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10 Of 26 


Questions 10-11 are based on the following teaching scenario. 

Mr. Foreman is teaching his sixth- and seventh-grade students a unit that 
focuses on strategies for reading informational text. His textbook includes a 
resource section with ideas that can help students identify the organization 
of text passages. The following graphic organizers are included in the 
textbook’s resource section: 


Description or List 


Compare and Contrast 

GD 


Assessment Tools 

1 

3 1 

► 

Hext 


- 


To best help his students improve their reading of informational text, Mr. 
Foreman should have them use these graphic organizers to 


O record and focus on the meaning of key vocabulary' in the text 

O comprehend that nonfiction texts are used to extract factual 
information 

O identify' concrete visual images to recall important details 

O anticipate patterns of information by identifying commonly used 
structures 


Sequence/Time order 


Cause and Effect 


(ZMD-© O 


Mr. Foreman is considering how' to use these examples to help his students 
understand how' graphic organizers can support their comprehension of 
informational text. 


zi 


Figure DIO Question #10. 
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Assessment Tools 
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Questions 10-11 are based on the following teaching scenario 

\ir. Foreman is teaching his sixth- and seventh-grade students a unit that 
focuses on strategies for reading informational text. His textbook includes a 
resource section with ideas that can help students identify the organization 
of text passages. The following graphic organizers are included in the 
textbook's resource section: 


Description or List 


Seijuence/Time order 

0-0-0 Q 


Compare and Contrast 

GD 


Cause and Effect 


Mr. Foreman is considering how' to use these examples to help his students 
understand how- graphic organizers can support their comprehension of 
informational text. 


Mr. Foreman is thinking about helpful information to share with his students 
about these graphic organizers. 

For each claim, indicate whether it is an accurate statement. 



Accurate 

statement 

Not an 

accurate 

statement 

"Cause and Effect" cannot always be used 
because effects sometimes precede causes 
in an essay. 



Sometimes more than one graphic organizer 
can be used because some essays have 
multiple text structures. 



"Compare and Contrast" is the most useful 
graphic organizer for nonfiction. 



There are more nonfiction text structures 
than those presented by these graphic 
organizers. 



"Description or List" and "Sequence Time 
Order" are always chronological. 



These graphic organizers can be used 
interchangeably. 




Figure Dll Question #11. 


Task 

Question Number 


Assessment Tools 

MET1 

12 of 26 


1 

1 ► 

| Hex 


A small group of students in Ms. Varga’s advanced ninth-grade English class are reading a book of historical fiction. She 
w-ants the students to develop more sophisticated interpretations of the text. To assess their understanding of the text, she 
gives the following wiiting assignment. 


Choose a character from the novel and wiite a journal entry that relates the events of the plot from the perspective of 
your chosen character. 

Be sure to: 

• accurately reflect the author's portrayal of the character in the novel 

• use specific details from the novel to bring your character to life 


Explain two ways that this assignment will help Ms. Varga evaluate her students' abilities to interpret the text. 


Figure D12 Question #12. 
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Assessment Tools 
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i 
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Questions 13-17 refer to the following teaching scenario 

Students in Ms. Joyce's ninth-grade classroom are reading Kurt Vonnegut's 
short story "Harrison Bergeron" as part of a unit on science fiction. 
"Harrison Bergeron" is a classic satirical tale written with a heavy dose of 
irony. In the story, set in 2081, "everyone was finally equal" because the 
"Handicapper General" has handicapped the more intelligent, athletic, or 
beautiful members of society. Ms. Joyce is using "Hamson Bergeron" to 
teach a lesson on irony. 

Ms. Joyce anticipates that many of her students will have trouble identifying 
irony in the story. To help them with this skill, she plans to use the opening 
paragraph of the story to identify features of Vonnegut's writing that signal 
to the reader that his story should be read as irony. She reads the opening 
paragraph aloud: 


THE YEAR WAS 2081, and everybody was finally equal. They weren't 
only equal before God and the law. They were equal every which way. 
Nobody was smarter than anybody else. Nobody was better looking 
than anybody else. Nobody was stronger or quicker than anybody else. 
All this equality was due to the 211th, 212th, and 213th Amendments to 
the Constitution, and to the unceasing vigilance of agents of the United 
States Handicapper General. 


Ms. Joyce wants to launch an introductory' discussion about Vonnegut's use of 
irony in the paragraph. 

Which of the following classroom discussion topics would best help her 
students focus on Vonnegut's use of irony in the paragraph? 

O Their understanding of constitutional amendments 

O Their ideas about the use of "God and the law" in the passage 

O Their thoughts on why science fiction is a good genre for expressing 
irony 

O Their perceptions of the meaning of equality' in the passage 




Figure D13 Question #13. Adapted from Welcome to the Monkey House by Kurt Vonnegut, Jr. Copyright © 1961 by Kurt Vonnegut, 
Jr. Used by permission of Dell Publishing, a division of Random House, Inc. 
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Questions 13-17 refer to the following teaching scenario. 

Students in Ms. Joyce's ninth-grade classroom are reading Kurt Vonnegut's 
short story "Harrison Bergeron" as part of a unit on science fiction. 
"Harrison Bergeron" is a classic satirical tale written with a heavy dose of 
irony. In the story, set in 2081, "everyone was finally equal" because the 
"Handicapper General" has handicapped the more intelligent, athletic, or 
beautiful members of society. Ms. Joyce is using "Harrison Bergeron" to 
teach a lesson on irony. 


Which of the following literary devices should the class focus on as the 
primary’ signal that Vonnegut is being ironic in the paragraph 0 

o Foreshadowing 
O Hyperbole 
O Repetition 
<C> Humor 


Ms. Joyce anticipates that many of her students will have trouble identifying 
irony in the story. To help them with this skill, she plans to use the opening 
paragraph of the story to identify' features of Vonnegut's writing that signal 
to the reader that his story should be read as irony. She reads the opening 
paragraph aloud: 


THE YEAR WAS 2081, and everybody was finally' equal. They weren’t 
only equal before God and the law. They were equal every' which way. 
Nobody was smarter than anybody else. Nobody was better looking 
than anybody else. Nobody was stronger or quicker than anybody else. 
All this equality' was due to the 211th, 212th, and 213th Amendments to 
the Constitution, and to the unceasing vigilance of agents of the United 
States Handicapper General. 


■ 

Figure D14 Question #14. Adapted from Welcome to the Monkey House by Kurt Vonnegut, Jr. Copyright © 1961 by Kurt Vonnegut, 
Jr. Used by permission of Dell Publishing, a division of Random House, Inc. 
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Questions 13-17 refer to the following teaching scenario 

Students in Ms. Joyce's ninth-grade classroom are reading Kurt Vonnegut’s 
short story "Harrison Bergeron" as pan of a unit on science fiction. 
"Harrison Bergeron" is a classic satirical tale written with a heavy dose of 
irony. In the story, set in 2081, "everyone was finally equal" because the 
"Handicapper General’ has handicapped the more intelligent, athletic, or 
beautiftil members of society. Ms. Joyce is using "Harrison Bergeron" to 
teach a lesson on irony. 

Ms. Joyce anticipates that many of her students will have trouble identifying 
irony in the story. To help them with this skill, she plans to use the opening 
paragraph of the story to identify features of Yonnegut's writing that signal 
to the reader that his story should be read as irony. She reads the opening 
paragraph aloud: 


THE YEAR WAS 2081, and everybody was finally equal They weren't 
only equal before God and the law. They w ere equal every w'hich wav'. 
Nobody was smarter than anybody else. Nobody was better looking 
than anybody else. Nobody was stronger or quicker than anybody else. 
All this equality was due to the 211th, 212th, and 213th Amendments to 
the Constitution, and to the unceasing vigilance of agents of the United 
States Handicapper General. 


After reading the paragraph from "Harrison Bergeron," Ms. Joyce asks 
students to write down w-hat they think the story is going to be about. She 
stops at one table and looks over the following written responses from three 
students. 

Akeem writes: This is going to be a science fiction story in which everybody 
is better oft 1 because everybody is equal. I am not sure how- everybody got to 
be equal in this story. But you can tell that the author thinks it is a good thing 
because he says everybody w-as finally equal. 

Lea writes: This story is going to make fun of the idea that everybody could 
ever be really equal. It's like when we say everybody is above average. That is 
just silly. So we know that the author must be kidding. 

Sarah writes: This is a story about the future. I really like thinking about the 
future and hoping that everything will be better then. I have never really 
thought about whether people will be more equal in the future than they are 
now. I usually just think about pollution and crime and war. Thinking about 
these things makes me pretty depressed. 

Rank the students listed below according to the level of understanding. 

To move each student's name to the appropriate box, click on the name, and 
then click on an empty' box. If you change your mind, click on the blue "undo" 
icon. Scroll down to see the boxes. 


Akeem 

Student 


Lea Sarah 

Level of understanding 


Best understanding 


Next best understanding 


Most limited understanding 


Figure D15 Question #15. Adapted from Welcome to the Monkey House by Kurt Vonnegut, Jr. Copyright © 1961 by Kurt Vonnegut, 
Jr. Used by permission of Dell Publishing, a division of Random House, Inc. 
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Questions 13-17 refer to the following teaching scenario After reading the paragraph from "Harrison Bergeron," Ms. Joyce asks 

students to write down what thev think the storv is going to be about. She 

Students in Ms. Joyce's ninth-grade classroom are reading Kurt Yonnegut's stops at one ubk ^ over the following written responses from three 
short story "Harrison Bergeron’ as part of a unit on science fiction. students. 

"Harrison Bergeron" is a classic satirical tale written with a heavy dose of 

irony. In the story, set in 2081, Everyone was finally' equal" because the Akeem writes: This is going to be a science fiction storv in which everybody 

"Handicapper General" has handicapped the more intelligent, athletic, or IS better off because everybody is equal. I am not sure tww everybody got to' 

beautiful members of society. Ms. Joyce is using "Harrison Bergeron" to be equal in this story. But you can tell that the author thinks it is a good thing 

teach a lesson on irony. because he says everybody w’as finally equal. 

Ms. Joyce anticipates that many of her students will have trouble identifying Lea ^tes: This story- is going to make fun of the idea that everybody could 

irony in the story. To help them with this skill, she plans to use the opening ever really' equal. It’s like when we say everybody is above average. That is 

paragraph of the story to identify- features of Yonnegut's writing that signal j ust S1 ]jy $o we know that the author must be kidding, 
to the reader that his story should be read as irony. She reads the opening 

paragraph aloud: Sarah writes: This is a story about the ftiture. I really like thinking about the 

ftiture and hoping that everything win be better then. I have never really- 
thought about whether people win be more equal in the ftiture than they’ are 
now. I usually-just think about pollution and crime and war. Thinking about 
these things makes me pretty depressed. 

For each student, select one term that best describes his or her response to 
Yonnegut's paragraph. 



Literal 

Interpretive 

Personal 

Analytic 

Akeem 





Lea 





Sarah 






THE YEAR WAS 2081, and everybody w-as finally equal. They- weren't 
only equal before God and the law. They were equal every which w-ay. 
Nobody was smarter than anybody else. Nobody- w-as better looking 
than anybody else. Nobody was stronger or quicker than anybody else. 
AQ this equality* was due to the 211th, 212th, and 213th Amendments to 
the Constitution, and to the unceasing vigilance of agents of the United 
States Handicapper General. 


Figure D16 Question #16. Adapted from Welcome to the Monkey House by Kurt Vonnegut, Jr. Copyright © 1961 by Kurt Vonnegut, 
Jr. Used by permission of Dell Publishing, a division of Random House, Inc. 
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Questions 13-17 refer to the following teaching scenario. 

Students in Ms. Joyce's ninth-grade classroom are reading Kurt Vonnegut's 
short story "Harrison Bergeron" as part of a unit on science fiction. 
"Hamson Bergeron" is a classic satirical tale written with a heavy dose of 
irony. In the story, set in 2081, "everyone was finally equal" because the 
"Handieapper General" has handicapped the more intelligent, athletic, or 
beautiful members of society. Ms. Joyce is using "Harrison Bergeron" to 
teach a lesson on irony. 

Ms. Joyce anticipates that many of her students will have trouble identifying 
irony in the story. To help them with this skill, she plans to use the opening 
paragraph of the story to identify features of Vonnegut's writing that signal 
to the reader that his story should be read as irony. She reads the opening 
paragraph aloud: 


THE YEAR WAS 2081, and everybody was finally equal. They weren't 
only equal before God and the law. They were equal every which way. 
Nobody was smarter than anybody else. Nobody was better looking 
than anybody else. Nobody was stronger or quicker than anybody else. 
All this equality’ was due to the 211th, 212th, and 213th Amendments to 
the Constitution, and to the unceasing vigilance of agents of the United 
States Handieapper General. 


ASS 

essment' 

fools | 





mL 1 


Akeem is a new student in the class. Ms. Joyce knows that he is still mastering 
English, and she wants to understand his thinking in order to draw him into the 
class's work. She reviews Akeem's response one more time before considering 
some possible interpretations. 


Akeem writes: This is going to be a science fiction story in which 
everybody is better off because everybody is equal. I am not sure how 
everybody got to be equal in this story’. But you can tell that the author 
thinks it is a good thing because he says everybody was finally’ equal. 


The most likely interpretation is that Akeem 

CD is confusing the narrator’s voice with the author's intent 

O does not understand what characterizes different genres 

O does not understand the vocabulary’ in the paragraph 

CD has identified the story as science fiction and reads it through that lens 
alone 




Figure D17 Question #17. Adapted from Welcome to the Monkey House by Kurt Vonnegut, Jr. Copyright © 1961 by Kurt Vonnegut, 
Jr. Used by permission of Dell Publishing, a division of Random House, Inc. 
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The ninth-grade students in Ms. Jett's class are writing short stories. The 
class has been learning to write concisely, including only’ information that is 
relevant to the author's purpose. Johanna chooses to write a suspense story. 
She starts her story’ with the following description: 

I w’as in a state of complete unaw-areness and becoming w’eary. Every 
tree looked the same, repeating itself over and over again, moving 
slightly as if a flip book. The path w’as long gone and now’ my purpose 
seemed silly. Earlier the rustle of a squirrel had frightened me but now’ it 
w’as background music and completely insignificant. 

The only reason I was there w’as that Lori—who has everything—had 
told me she w’ould give me her allow’ance for five w’eeks. Lori's 
allow’ance is three times as big as mine and her closet is full of pretty 
dresses with sequins that sparkle like stars. I had my eye on the perfect 
prom dress but it was too expensive. Lori's allow’ance w’ould just pay 
for it, if I ever got back. 

Everyone always told me dares were childish and as productive as 
hitting your head against a walL My friends had told me I was crazy to 
be going to Mr. Bloom's w’oods. Everybody w’hispered that he w’as 
strange, and may have done something bad in the past. Mom wouldn't 
have put her stamp of approval on this little adventure, but what she 
didn't know’ wouldn't hurt her. But I was beginning to wonder if it 
w’ould hurt me. 


Assessment Tools 


I ► 

J_1 Hex 


Ms. Jett conferences with Johanna to discuss the beginning of her story. 

What two suggestions about the second paragraph should Ms. Jett provide to 
help Johanna improve her story? 


Figure D18 Question #18. 
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Ms. Rice begins a unit on memoir writing by reading a passage from a literary model. She then asks students to complete a warm¬ 
up activity to help them generate ideas for their own writing. 


For each assignment, indicate whether it will help students focus their brainstorming on generating a memoir. 



Will help focus 
brainstorming 

Will not help focus 
brainstorming 

Write a poem about the the ways you have changed, using the form "I 
used to be ...but now I am..." 



Write a sequence of sentences describing some of vour experiences, 
beginning each sentence with the phrase "I remember." 



Write a few’ adjectives that describe your personality. 



Write down some of vour favorite foods and describe w’hat you like about 
them. 




Figure D19 Question #19. 
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Ms. Gupta has been working with her eighth-grade students on writing Which of these follow-up responses from Ms. Gupta would best help Jacob 

memoirs that focus on a single important event. One student, Jacob, tends develop a stronger memoir? 

to struggle with writing. Ms. Gupta is pleased to see him working 
enthusiastically on his memoir about going to a NASCAR race. In a one-on- O 

one conference, Ms. Gupta reads Jacob's opening paragraph and recognizes 
that it represents a significant advance in his writing: 


o 

o 




The sky looked like fireworks bursting in a combination of 
colors: red and blue, green and yellow’, pink and green, green and 
orange, orange and purple, purple and white. The clouds looked 
like a bunch of exotic cars breezing by my house, with their fully 
customized bodies. The sun looked like a great big bubble 
popping in the sky. When the cars started to drive by the pit lane, 
they w’ere like a group of lazy turtles until the green flag w’ent 
out. Then they started to move like a group of forty-three road 
runners. 


"I like your vivid use of colors at the beginning. Can you use other 
details like that to describe vour setting?" 

"I like the way you bring the reader right into the middle of the scene. 
Can you tell me more about w’hat you w’ere doing w’hile all of this w-as 
happening?" 

"I like how’ you use all those similes to really describe w’hat you're 
seeing. Can you supplement them with some metaphors?" 

"I like the w’av you show’ the reader w’hy you think car racing is so 
exciting. Can you use dialogue to show’ the crowd's excitement?" 


Figure D20 Question #20. 
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MET1 

21 of 26 


1 

” | 




Questions 21 and 22 are based on a writing assignment given to Mr. Johnson’s class. 

The students in Mr. Johnson's class are writing the first draft of an essay about someone who has influenced them. Mr. Johnson wants each student's essay to have a 
clear thesis statement and supporting details. He explains, "Your thesis statement should tell why the person has influenced you, and the rest of your essay should show 
me that whatever you said in your thesis statement is true about that person." 

Mr. Johnson circulates around the room and checks students' writing. He looks over the following initial draft from one of his students, named Frank. 

Why My Dad is Cool 
by Frank 

The person who has influenced me is my father because he smart, fun, an nice person. The reason he influenced 
because he always telling me that you can do it and no one can stop you But you he also let he come and vist him 
anytime. He support me. 


Indicate whether Frank's essay has met each of the following objectives from Mr. Johnson's assignment. 



Has met objective 

Has not met objective 

Write a clear thesis statement that teUs why someone has influenced you. 



Include supporting details about why someone has influenced you. 



Align your thesis statement with supporting details. 




Figure D21 Question #21. 
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Question Number 


Assessment Tools 

MET1 

22 of 26 




J!. 


Questions 21 and 22 are based on a writing assignment given to Mr. Johnson’s class. 

After receiving a few comments from Mr. Johnson, Frank produces the following second draft. 


Why My Dad is Cool 
by Frank 

The person who has influenced me is my father because he smart, fun, an nice person. The reason he influences 
because he always telling me that you can do it and no one can stop you But you he also let me come and vist him 
anytime I get ready Another reason is because he The way he dress is like they do in the army he is 6 8 weigh about 
195 and a basketball player. One day me and him play one on one he use to play ball. He tell me that you can become 
the best player then any body One day he show me how to play the game and how to post up to other players and 
dunk. He went to school at Broad Street High School were he won the first gold ball for Broad Street he was a strong 
A's student from High School to collage were he won the gold ball at FCULA collage in California were he won another 
gold ball 1987. 


Mr. Johnson realizes that Frank is having trouble with this assignment. He wants to select a focus for Frank's next revision to help him improve the quality of his 
essay. 

Which of the following would be the most important for Frank to address in his next revision? 

O Spelling and punctuation 
O Grammar and verb tense 
O Staying on topic 
O Organization of ideas 

j 


Figure D22 Question #22. 
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Task 

Question Number 

MET1 

23 Of 26 



Mr. Jackson's sixth- and seventh-grade students are writing flash fiction. After reading the following draft of Arielle's short story, 
Mr. Jackson wants to point out some of the strengths in her writing. 

Natasha left the house and walked sadly down the sidewalk, slowly shuffling her feet. As she walked by the old house, she 
was surprised by how sad she felt. She stepped onto the grass, and the sprinklers sprayed her face. The sprinkle of cold water 
caught her by surprise. Her brother shouted down the street asking her to wait up. Thankfully he couldn’t tell the tears from 
the drops of water. 


For each writing feature, indicate whether it describes a strength in Arielle's writing. 



Describes a strength 
in .Arielle's writing 

Does not describe a strength 
in Arielle’s writing 

Strong character development 



Use of figurative language 



Conveying emotions through showing 



Use of powerful images 




Figure D23 Question #23. 
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24 of 26 



Assessment Tools 


► 
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Questions 24-25 refer to the follow ing teaching scenario 

Mr. Perez is preparing a lesson that requires his seventh-grade students to 
read an informational text in their English anthology about the sinking of the 
Titanic. The text is an excerpt from Exploring the Titanic by Robert Ballard, 
an example of literary nonfiction that uses an informational narrative format. 
Instructional activities in the anthology focus on distinguishing between fact 
and opinion. Although Mr. Perez thinks this is a useful skill, he wants his 
students to think more deeply about the craft of writing informational text. 
He brainstorms a list of possible activities to help students increase their 
knowledge of narrative features of nonfiction text. 


For each student activity, indicate whether it is likely to help Mr. Perez's 
students increase their knowledge of narrative features of this informational 
text. 



Likely to 
help 

Not likely 
to help 

Describing how the author uses sequential text 
structure to tell the story of the Titanic 



Identifying and tracking unfamiliar naval vocabulary 
words and trying to determine their meaning from 
context clues 



Analyzing how the author uses a description of the 
Titanic to reflect the social values of the early 1900s 



Explaining why the author chooses eyewitness 
accounts to describe what happened during the 
sinking of the Titanic 




Figure D24 Question #24. 
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Questions 24-25 refer to the following teaching scenario Mr. Perez asks his students to analyze the following paragraph from the book: 

Mr. Perez is preparing a lesson that requires his seventh-grade students to 

read an mformational text m then English anthology about the smiting of the For ntxt months Tiamc was outfit,td "* P rt P artd 

Titanic. The text is an excerpt from Exploring the Titanic by Robert Ballard. down ,0 *' ^ Thc fcal md nchn ' ss of ** new 5hi P was 

an example of Uterarv nonfiction that uses an mformafional narrative format. mounding. She was 882 feet long, almost the length of four city blocks. 

Instructional activities m the anthology focus on dtstingutshmg between fact With nm5 dfcks - she was as high as an eleven-story butidmg. 

and opinion. Although Mr. Perez thinks this is a useful skill, he wants his 

students to think more deeply about the craft of writing informational text. j n keeping with his goal of focusing students’ attention on the craft of writing 

He brainstorms a list of possible activities to help students increase their informational text, which of the following questions should Mr. Perez ask the 

knowledge of narrative features of nonfiction text. class? 

O "How does the author help us understand the size of the Titanic?" 

O ’How long do you think it would take to build the Titanic today?" 

O "How do the details in this paragraph help the author emphasize the 
difficulties in building the Titanic?" 

O "How would you describe the size of the Titanic?" 


2 


Figure D25 Question #25. 
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26 of 26 


1 

I 

► 
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Ms. Lee is teaching a unit on persuasion to her sixth- and seventh-grade students. Although the students have read articles 
on the pros and cons of their topic—eliminating vending machines from school cafeterias—she notices that they have not 
used textual evidence in their essays. 

Which of the following is the most effective activity to help Ms. Lee’s students add textual evidence to their essays to 
support their arguments? 

Ms. Lee should ask her students to 

O critically read articles about the pros and cons of having vending machines in schools 

O record evidence they find in articles on sticky notes and place the sticky notes in the appropriate places in their 
drafts 

O peer edit each others' drafts with a focus on identifying both strengths and weaknesses in the essays 
O use *eir writing guides to apply the proper citation style for authoritative sources 


Figure D26 Question #26. 
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Appendix E 
Algebra I Assessment 


Table El Item/Question Sequence Number for Algebra I Assessment 


Item/question sequence number 

Answer key 

Item/question sequence number 

Answer key 

1 

B 

lid 

Valid 

2 

C 

lie 

Not valid 

3a 

Provides 

12 

B 

3b 

Does not provide 

13 

B 

3c 

Provides 

14 

C 

3d 

Does not provide 

15a 

Provides 

3e 

Provides 

15b 

Provides 

4 

C 

15c 

Provides 

5 

C 

15d 

Does not provide 

6a 

Would not support 

15e 

Does not provide 

6b 

Would support 

16 

C 

6c 

Would support 

17 

C 

7 

B 

18 

B 

8 

C 

19 

D 

9 

C 

20 

C 

10 

B 

21 

D 

11a 

Valid 

22 

D 

lib 

Valid 

23 

C 

11c 

Not valid 





Question Number 
1 of 23 


Assessment Tools 


► 

Next 


Mr. Wright asked his students to solve the equation 6 - 3(x - 5) - 24. After reviewing his students' work, he found two 
interesting methods for solving the equation and asked those two students to present their methods on the board. 


Brenda's method 

6 - 3(x - 5) - 24 
6 - 3x + 15 - 24 
-3x + 21-24 
-3x - 3 
x - -1 


Daniel's method 

6 - 3(x - 5) - 24 
-3(x-5)« 18 
x - 5 = -6 
x--l 


After Brenda and Daniel presented their methods, Mr. Wright's class discussed these two methods. One student, Steve, 
compared them and then said, "I like Daniel's method because there are less steps. However, if it is a harder division, 
Brenda's method would be easier." For which of the following equations would Steve be most likely to use Brenda’s 
method? 

O 5 - 2(x - 3) - 21 
O 10 - 7(x + 5) - 6 
O 6-3(x+l)-9 
O 8 + 4(x - 3) - 14 


Figure El Question#!. 
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Task || 

|| Question Number 


Assessment Tools 

MET1 

2 of 23 


4 1 

Back | 

1 

► 

Next 





A lesson in Ms. Hagerman’s textbook defines the distributive property, but the exercises merely ask for its definition. To 
motivate her students to learn the definition, Ms. Hagerman tells them that the distributive property can often be used to 
simplify' the evaluation of expressions. 

She wants to give her students an example that will focus their attention on how' the distributive property' can be useful in 
evaluating expressions. Of the following expressions, w’hich would best serve her purpose? 

ou-(H) 

° “*(}-&) 
o jHlrf) 

O Each of these expressions w'ould serve her purpose equally well. 


Figure E2 Question #2. 
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Task 

Question Number 


Assessment Tools 

MET1 

4 of 23 


4 1 

Back | 


► 

Hex' 



Mr. Browns tern's class was solving the equation: 

3(x - 2) 2 - 6(x - 2Xx + 5). 

Kenneth suggested dividing both sides by 3 to get: 

(x-2) 2 -2(i-2X* + 5). 

Then he suggested dividing both sides by (x - 2), but Sandra said, "You cannot divide both sides by (x - 2)." In response, 
Kenneth asked, "If you can divide both sides by 3. why can't you divide both sides by (x - 2)?" 

Of the following statements, which best explains whv you cannot divide both sides of the equation by (x - 2) as Kenneth 
suggested? 

<3 You cannot cancel (x - 2) because it represents a real number. 

O* It is better to expand the expressions on both sides of the equation first to obtain x 2 - 4x + 4 - 2(x 2 + 3x - 10), 
and then you won't have to worry about (x - 2). 

O Division by zero is not defined, so you w'ould have to consider the case of x - 2 separately. 

O Because x is a variable, it can vary—you may not be canceling the same amount from both sides. 


Figure E4 Question #4. 


Task || 

| Question Number 

MET 1 

5 of 23 



Assessment Tools 

Ba1k| 

Li | 

ul 


During a unit on solving linear equations, Ms. Martino asks her students to write one question that she could use on the unit 
test at the end of the chapter. Joe writes the following question. 


If Joe had to solve the sentence 8 y - 9 = 0 for y, what w'ould be the value of y ? 


Ms. Martino thinks the equation would be a good one for her students to solve. However, she decides to revise the question 
because she is concerned that the wording may cause some of her English Language Learner (ELL) students to answer this 
question incorrectly, even if they understand the mathematics involved. Of the following revisions, which one best 
addresses Ms. Martino's concern? 

O When Joe solved for y m the equation 8y -9 = 0, what was the value ofy ? 

O Joe solved the sentence Zy - 9 = 0 for y. \Miat is the value of y ? 

O What is the value of y in the equation 8y - 9 = 0 ? 

O If 8>’ - 9 = 0, what would be the value of y ? 


Figure E5 Question #5. 
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Task 

Question Number 

MET1 

6 of 23 




| | Assessment Tools 
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Before teaching a lesson on multiplying two trinomials, Ms. Ryan wants a better sense of what her students know about multiplying two binomials. She asks 
them to find the product (2x + IX* - 4) and explain their methods. While walking around the class, she notices several different methods. For each of the 
following, indicate whether or not the student response provides evidence that the student has an understanding of the multiplication of binomials that would 
support the development of a strategy for multiplying two trinomials. 


I used the FOIL method. The product of the first 

terms is 2x 2 , the product of the outer terms is -8x, 
the product of the inner terms is X, and the product 
of the last terms is -4. Then, I combined the two 
like terms, the -8x and the x, to get -7x, so my 
final answer is 2x 2 - 7x - 4. 



Ox* -7x-p 


I just multiplied like you do with regular numbers. I 
put the 2x + 1 on the top and the x - 4 on the 
bottom. Then I multiplied each term on the bottom 
by each one on the top. Finally, I added up the like 
terms to get my answer, 2x 2 - lx - 4. 


0?X +- / 

x-V 


-</>x 

y x 


-7* -v 


I drew’ a box with two columns for the 2x and the 1 
and two rows for the x and the -4. Then, I 
multiplied the terms that w'ent with the row' and 
column that each small box w’as in. Then, I wrote 
out w'hat I got in each small box, combined the like 
terms, and got 2x 2 - lx- 4. 


Px 



1 - 

X 

-A 1 

-V 


3 ** + 


Would Support a Strategy 
for Trinomials 


Would Not Support a Strategy 
for Trinomials 


Figure E6 Question #6. 
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Mr. Anderson asked his students to simplify the following algebraic expression. 

2(a + 1) _2_ 6a-2 

3 a + 3 3 a 6 


One of his students gave the incorrect solution shown to the left. 

Of the following descriptions, which best characterizes what is wrong with 
this student's work? 

O This student used the distributive property incorrectly. 

O This student confounded mixed fractions with factors. 

O This student forgot to cancel common factors in several places. 

O This student needs to apply a more formal procedure by finding the 
common denominator and then adding all terms. 


Figure E7 Question #7. 


Task 

Question Number 


| Assessment Tools 

MET1 

8 of 23 



4 1 

Back | 




Having nearly finished a chapter on linear equations, Mr. Hassan's students seem quite proficient in generating standard 
formats for linear equations, using techniques for graphing linear equations, and solving systems of two linear equations. 
However, he is concerned that his students are applying these techniques in routine ways and tend to think only 
algebraically or only geometrically without reasoning fluidly in both ways. 

Mr. Hassan wants to give his students a problem that would require an understanding of the topic that goes beyond the set 
of procedures students have learned and that would support his students' ability to work and talk across algebraic and 
geometric interpretations. Of the following problems, which would best serve this dual purpose? 

O Describe in your own words a procedure for finding the point of intersection given the equations of two lines. 

O Find the intersection of the following two lines and graph them. 
y-2x + 3 
>• - 2x - 7 

Consider two linear functions, where a and b are negative. 
y-x + 3 
y - ax + b 

What can you say about the point of intersection of their graphs? 

O Csing ideas about solving systems of two linear equations, solve the following system of three equations and 
explain what the solution means. 

x + 4y + z - 0 
x - 4y + 2z - 3 
x«4>+z 


Figure E8 Question #8. 
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9 of 23 


< I 

Back | 
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Having taught her students to factor quadratics with integer coefficients, integer roots, and a leading coefficient of 1, Ms. 

Quezada explained that she was going to give them a harder problem. She then asked them to solve the following. 

3x 2 - 3x - 6 - 0 

After a few minutes of work, the class discussed their solutions. Letitia said that x was -1 or 2 and explained, "I added 3x 
to both sides and divided by 3." 

3x 2 - 6 - 3x 
x 2 - 2 * x 

She then continued, "The parabola's just down a little and the line's at 45 degrees, so it's just below zero and about 2 to the 
right, x can be -1 and 2, and those are the only possible ones." 

Of the following, which best characterizes Letitia's approach to this problem? 

O Letitia's method is wrong because she should have first divided by 3 and then factored the left side of the equation. 

O Letitia's method is wrong because this is a parabola and you could graph it, but you would have to graph the original equation and 
look for the roots. 

O Letitia's reasoning is correct, but her method often leads to points of intersection that might be hard to determine visually. 

O Letitia's reasoning is correct, but her method requires knowledge of calculus. 


Figure E9 Question #9. 
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Ms. Lang’s class had been studying the concept of slopes of lines, so she asked them to consider all of the lines passing 
through one point and how the slopes of those lines vary. The students had used geoboards in some earlier work, so they 
started talking about the slopes of lines on an "infinitely extended" geoboard. (Geoboards are flat blocks of wood, roughly 
one foot square, with pegs laid out on a gnd where rubber bands can be hooked to make lines or polygons.) The students 
decided that the pegs of the infinite geoboard could be thought of as the set of points with integer coordinates in the 
Cartesian plane. 

During the discussion, students had the following exchange. 

Yonah: On the geoboard, you can't get all of the slopes, because the geoboard points are too spread out—there are a 
whole bunch of lines between the ones you can make. 

Andy: I disagree. I think we can make any slope. Starting at one point, by choosing another geoboard point far 

enough away, we can tilt the line as much or as little as we like. 

Becky: What I was thinking was if you run a line through one geoboard point, it will always hit another one far 

enough out. 


Of the following concepts, which is most directly related to the mathematics underlying this discussion? 

O Interpretation of the derivative—the derivative is the slope of the tangent line. 

0> Density of numbers on the real line—the rational numbers are dense, but not every real number is rational. 

O The parallel postulate—given a point and a line, there is a unique line through the given point parallel to the given 
line. 

O Each of these concepts is equallv related to the underiving mathematics. 

_ a 


Figure El0 Question #10. 
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During a lesson on solving multistep equations, Mr. Steinbrecher asked his students to solve the equation 4(5x - 11) - 16. 
While walking around the class looking at what the students were writing, he noticed several different strategies. For each 
of the following student solutions, indicate whether or not it is a valid strategy for solving this problem. 



Strategy Is Valid 

Strategy Is Not Valid 

Ta-II z V 

S’yn /r 

<nx> 



v. y(fx-n) = it ,-h 

Htfi’"-") * Xr-t 

yi ~V 

Si-n = v 
til fl1 

-- /S" 



‘/(S*-//) - It, 

fx O' 

/1 +»_ 

rr 

CETD 



Wf'Tx-il) =11, 

- VV = It, 

Po Jo 

t+L 

Jo 

*'g 



It, 

t * * H 

+jq 

' i 

y( gx t£ . >L 
tf ' X * , 

<*EX> 




Figure Ell Question #11. 
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Back 
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Mr. Jakobsen's students were graphing the function below, where y is inversely proportional to x. 



One of his students drew the following graph. 



Mr. Jakobsen has noticed that students often draw graphs with line segments like this despite frequent reminders that the graph should be curved. To get his 
students to discuss this issue, he asked the class what was wrong with the drawing. Of the following student explanations, winch provides the best 
mathematical explanation of why drawing connected line segments is inappropriate for this graph? 

CD When the x changes, the graph should change at the same rate all the time and it shouldn't have comers. 

CD The graph changes all the time, but it cannot have sudden changes at some of the points. 

CD For any w’hole number, will always be a rational number and that makes it hard to draw’ the graph for irrational numbers. 

O The problem is that you don't have enough points. You need to include more points to make it look correct. 


Figure El2 Question #12. 
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In a unit on simplifying expressions, one of Mr. Serrano's students wrote 
the following correct solution. 

■*(■“*») , j. » 6q~* 

3a 3a 6 


be I 

- , 

6 » (j> 




3<9 

^ 3 * 


1 (o 

2a + 



I 7 

/ 3 * 

Iz-* 


l 7 

lx + 

lz ~ « 

at 

- 0 

3i 

- a 

■ 3 j 

- <3 


± 


Of the following descriptions, w’hich best characterizes the student's w’ork? 

O The student know’s how’ to simplify expressions very w’ell and 
demonstrates strategic use of standard procedures. 

O The student show’s good computational skill but does not use 
processes efficiently. 

CD The student know’s how to simplify expressions very w’eU, but in the 
solution the student should wTite all steps involved in the calculation, 

8 8 

such as the step ---—. 

v 3 a 3 a 

O The student should apply a more formal procedure by first finding the 
common denominator and then adding all terms. 


Figure El3 Question #13. 
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A lesson m Ms. Taylor's textbook states the associative and commutative properties of addition. To motivate the students 
to learn the properties, she tells her students that the properties can often be used to simplify the evaluation of expressions. 

She wants to give her students an example that will focus their attention on how' these properties can be useful in evaluating 
expressions. Of the following expressions, w'hich would best serve her purpose? 

O (455+ 456 > + (457 + 458) 

O (647 + 373 > + (227 + 456) 

O ( 551 + 775 > + ( 49 + 225 ) 

O Each of these expressions w'ould serve her purpose equally well. 


Figure E14 Question #14. 
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Ms. Kamp asks her students to consider squares of different side lengths with only the boxes along the sides shaded as in the figure 
on the left. 

She asks each student to wiite an expression for the number of shaded boxes in a square with a side length of n boxes and to 
explain why the expression gives the number of shaded boxes for any size square. For each of the following explanations, indicate 
whether or not it provides evidence that the student understands whv the expression can be used to find the number of such shaded 



Provides 

Evidence 

Does Not Provide 

Evidence 

If you start on the bottom left and go to just below' the top left, then start with the top left and go to just before the top 
right, and keep doing that, you will get 4 groups, and each group has 1 less than n, so you get 4(« - 1). 



My expression is n + 2(n - 1) + (n - 2) because the top of the square has n shaded boxes, then each of the sides has 
n - 1 shaded boxes left, and then the bottom has n - 2 shaded boxes left. 



Inside the square with a side length of n boxes is a square with side length of n - 2 boxes, so if you find the area of the 
two squares and subtract them, you will find the number of shaded boxes. So, I get n 2 - (n - 2) 2 . 



I get 4(n - 2) + 4 because there are 36 boxes shaded, and when you put 10 in for the n in 4 (h - 2) + 4 and follow' the 
order of operations, the answer is 36. 



My expression is In + 2(« - 2) because if you simplify 2 n + 2(n - 2) you get 2n + 2n - 4, which is equal to An - 4. 
and because this doesn't depend on n, it w'orks for any n. 




Figure El5 Question #15. 
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Ms. Quinn asked her students to solve the following quadratic equation. 

3x 2 - 6x - 24 = 0 

Maurice explained, "I added 24 to both sides and divided by 3." 

3x 2 - 6x = 24 
x 2 - 2x - 8 
x(x - 2) * 8 

He then concluded, "The only numbers that are 2 apart and multiply to be 8 are 2 and 4, and -2 and -4, so x has to be 4 or 
-2." Students agreed that 4 and -2 work when you substitute them into the original equation, but they were unsure about 
his method. 

Of the following statements, which best characterizes Maurice's approach to this problem? 

O Maurice’s method is wTong because you cannot solve an equation by factoring unless one side of the equation is equal to zero. 

O Maurice's method is wxong because he should have first divided by 3 and then factored the left side of the equation. 

O Maurice's reasoning is correct , but his method often leads to an equation that cannot be solved by inspection. 

O Maurice's reasoning is correct , but his method only works for equations with real roots. 


Figure El6 Question #16. 


Task || 

|| Question Number 


Assessment Tools 

MET 1 

17 of 23 


4 1 

Back| 


► 

Nex 



In the last class, Mr. Rosen's students graphed quadratics of the form y - x 2 + c for various values of c and developed a rule about shifting the graph of y - x 2 
up or dowTi. Today Mr. Rosen asked the students to predict what would happen when they graphed y - (x - 3)~ and then to graph it. Students w r ere surprised 
that the graph shifted 3 units to the right rather than left or down as they had predicted. Mr. Rosen asked them to explore a little further in groups. 


As he w-alked around the classroom, each group explained to him the strategy they w-ere using to explore the problem. Of the following student descriptions of 
a strategy for exploring the problem, w’hich is most directly related to the underlying mathematical reason for the graph's behavior? 


O We are trying to prove the rule, so each of us is graphing another one, y - (x - 2) 2 , y - (x - 5) 2 , y - (x + 2) 2 , and y 
compare our results. 


(x + l) 2 and then we will 


o 


We are making a table like yesterday and putting x and x l and (x - 3)*, so we can plug in different inputs and compare w’hat the outputs are in x‘ and 
(x-3) 2 . 





X 

X 2 

(x-3) 2 

0 



1 







O We decided to look at the roots ofy ■ x 2 and y * (x - 3) 2 . The vertex of y = (x - 3) 2 is (3, 0), and the vertex of y * x 2 is (0, 0). We are looking to 
see w’hat x-vahies w f e have to put in (x - 3) 2 to make the outputs the same as in x 2 . 

O We are going to FOIL it out so it will look more like the ones from yesterday, and then w’e can graph it and compare the way w r e did yesterday. 


Figure El7 Question #17. 
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Ms. Seidel is introducing the distributive property. To motivate her students, she wants to give them an example that will 

focus their attention on how using the distributive property can simplify computations. In which of the following examples 

will the use of the distributive property most simplify the computations? 

O 12 * 29 + 12 x 38 -_ 

O 17 x 37 + 17 x 63 =_ 

O 13 x 13 + 15 x 15 =_ 

O 16 x 24 + 16 x 24 -_ 


Figure El8 Question #18. 
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Ms. Christensen asked Carla to simplify the following expression. 

10a + 4 
2 a 

Carla wrote the following incorrect solution. 

10 *+ ^ 

fit 

Ms. Christensen then asked the class what was wrong with this solution. Which of the following student explanations 
characterizes what w r as most likely wTong with Carla's solution? 

O She should have wiitten 2(5a + 2 ) in the top and then canceled the 2 s in the top and the bottom. 

O She saw r you can break the fraction into two fractions, but the way she simplified each fraction is WTong. 

O She divided the 4 by 2 , but you cannot cancel w’hen you have more than one thing added in the top. 

O It's not possible to cancel like this because you can only cancel factors that are the same for the top and the 
bottom. 


Figure El9 Question #19. 
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Ms. Lindsey's textbook uses a geoboard to model slope as rise over run. (Geoboards are flat blocks of wood, roughly one 
foot square, with pegs laid out on a grid where rubber bands can be hooked to make lines or polygons.) As her students 
explored different slopes they could make on the geoboard, Edward asked, "Since the diagonal of one of the unit squares 
has length V 2 , does that mean you can make a line segment with slope \ 2 on the geoboard?" When Ms. Lindsey asked 
the class whether they thought this could be done, the following exchange occurred. 

Andy: Edward's right that the diagonal of the unit square has length V 2 , but its slope is 1. 

Beth: Well, that doesn't matter. We can just turn the geoboard so that the diagonal is horizontal, and then we can 

see squares with side length \ 2 . 

Caitlin: Sure, but the square roots of two would just cancel. I think they always would, so you can't get ^ 2 as a 
slope. 

Dan: That's not right, because we can make one length of ^ 2 and another length of 1 and use them as the rise 

and the run. 

Which of the student statements gives the best insight into Edw’ard's question? 

O Andy's statement 
O Beth's statement 
O Caitlin's statement 
O Dan's statement 


Figure E20 Question #20. 
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Mr. Roberts' class has been learning to factor quadratics. His students know' that the polynomial x 2 - 4x + 3 can be 
factored into (x - 3X* - 1), and he wants to build on this knowledge to factor special cases of higher-degree polynomials 
that are quadratic in form. He starts off with x 4 - 4x 2 + 3 - ( ) ( ) and asks his students to suggest pow'ers of x that can 
be put into each set of parentheses. 

Carmen suggests that the first term in the first set of parentheses could be x 3 , and the first term in the second set could 
be x. Of the following statements, w’hich best characterizes Carmen's proposal? 

O Her idea for factoring the quartic will not w’ork, because no matter how* you complete the factors, w'hen you 
multiply it out, you'll end up with an x 3 . 

O Her idea for factoring the quartic will not w'ork, because the assumption in this context is that the polynomials are 
over the real numbers, not over the complex numbers. 

O Her idea for factoring the quartic will w'ork, but you have to choose the other two numbers correctly. If you use 
-3 and -1, when you multiply it out, you get x 4 and -1 times -3 equals 3. 

O Her idea for factoring the quartic will w'ork, but it produces a factorization that is not of the same form as the 
quadratics that the class has already learned to factor. 


Figure E21 Question #21. 
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Ms. Collingwood is teaching a unit on graphing. Some of the students in her class speak a language other than English at 
home and when they work in small groups. As she prepares for the unit, she makes a note of vocabulary words that she 
believes will be challenging for her students, especially words that have different meanings in different contexts. Of the 
following vocabulary words that she identified, which will require the least clarification regarding differences in meaning? 

O Plane 

O Origin 
O Intercept 
O Parabola 


Figure E22 Question #22. 
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Mr. Baas' students were solving inequalities. Cheryl wrote the following solution on the board. 

^—=•<1 

x 

x -2 < x 
0 < 2 

She concluded that because this is always true, every x w ould work. After some discussion, students decided that Cheryl's 
solution w-as not correct, but they w-ere unsure w'hy. Of the following explanations, w'hich best identifies w’hat is 
problematic about Cheryl's w’ork on the problem? 


<3 It's true that 2 is always greater than 0, but because you have eliminated all x, we cannot say w’hat x is. 

O c can see that the numerator always is two less than the denominator, so the fraction will ahvavs be less than 1 
for all x. However, we have to require the denominator x ? 0. 

O You don't know* w’hat x is, so w’hen you multiply' with x like this, you must assume x > 0. 

2 

O She should have simplified the left-hand side of the inequality to 1 - — and then subtracted 1 from both sides and 
added — to both sides. This w'ould vield 0 < —. w'hich is true for x > 0 . 

X X 


Figure E23 Question #23. 
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Appendix F 

Item Statistics for Content Knowledge for Teaching (CKT) Assessments 

Table FI Mathematics 4-5 Item Statistics 


Item/question sequence number 

P-plus value 

Biserial correlation 

1 

0.73 

0.49 

2 

0.56 

0.51 

3a 

0.78 

0.49 

3b 

0.65 

0.40 

3c 

0.73 

0.51 

3d 

0.67 

0.30 

3e 

0.59 

0.48 

4 

0.39 

0.58 

5 

0.71 

0.46 

6 

0.54 

0.57 

7a 

0.73 

0.19 

7b 

0.81 

0.44 

7c 

0.76 

0.52 

7d 

0.52 

0.25 

7e 

0.86 

0.43 

8 

0.29 

0.47 

9 

0.61 

0.39 

10a 

0.96 

0.47 

10b 

0.82 

0.31 

10c 

0.50 

0.46 

lOd 

0.47 

0.36 

lOe 

0.63 

0.11 

11 

0.42 

0.36 

12 

0.19 

0.47 

13 

0.33 

0.20 

14a 

0.46 

0.44 

14b 

0.81 

0.52 

14c 

0.43 

0.39 

14d 

0.83 

0.42 

14e 

0.57 

0.59 

15 

0.26 

0.50 

16 

0.23 

0.46 

17 

0.12 

0.56 

18 

0.13 

0.17 

19a 

0.74 

0.42 

19b 

0.56 

0.16 

19c 

0.56 

0.34 

19d 

0.28 

0.06 

20 

Constructed-response item 


21 

Constructed-response item 



Table F2 Mathematics 6-8 Item Statistics 

Item/question sequence number 

P-plus value 

Biserial correlation 

1 

0.58 

0.68 

2 

0.62 

0.42 

3a 

0.98 

0.35 

3b 

0.71 

0.53 

3c 

0.82 

0.59 

4 

0.33 

0.63 

5 

0.64 

0.31 

6a 

0.84 

0.36 

6b 

0.86 

0.62 

6c 

0.92 

0.70 

6d 

0.69 

0.45 
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Table F2 Contiuned 


Item/question sequence number 

P-plus value 

Biserial correlation 

6e 

0.91 

0.47 

7 

0.38 

0.38 

8a 

0.93 

0.67 

8b 

0.66 

0.58 

8c 

0.95 

0.56 

8d 

0.73 

0.37 

8e 

0.79 

0.41 

9 

0.57 

0.44 

10 

0.29 

0.37 

11a 

0.76 

0.58 

lib 

0.84 

0.54 

11c 

0.79 

0.61 

lid 

0.62 

0.47 

lie 

0.86 

0.45 

Ilf 

0.75 

0.35 

12 

0.39 

0.51 

13 

0.68 

0.57 

14a 

0.53 

0.51 

14b 

0.46 

0.57 

14c 

0.35 

0.28 

14d 

0.61 

0.32 

15 

0.27 

0.49 

16a 

0.74 

0.41 

16b 

0.72 

0.41 

16c 

0.65 

0.49 

16d 

0.56 

0.55 

16e 

0.43 

0.32 

17 

0.38 

0.22 

18 

0.21 

0.43 

19a 

0.42 

0.04 

19b 

0.95 

0.50 

19c 

0.86 

0.50 

19d 

0.61 

0.55 

20 

0.18 

0.34 

21 

0.09 

0.12 

22 

Constructed-response item 


23 

Constructed-response item 



Table F3 Algebra I Item Statistics 

Item/question sequence number 

P-plus value 

Biserial correlation 

1 

0.76 

0.53 

2 

0.55 

0.68 

3a 

0.98 

0.02 

3b 

1.00 

_a 

3c 

0.80 

0.58 

3d 

0.88 

0.61 

3e 

0.99 

0.71 

4 

0.77 

0.63 

5 

0.59 

0.23 

6a 

0.30 

0.50 

6b 

0.87 

0.41 

6c 

0.95 

0.16 

7 

0.43 

0.53 

8 

0.41 

0.57 
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Table F3 Continued 


Item/question sequence number 

P-plus value 

Biserial correlation 

9 

0.48 

0.55 

10 

0.31 

0.53 

11a 

0.97 

0.81 

lib 

0.68 

0.61 

11c 

0.98 

0.62 

lid 

0.84 

0.55 

lie 

0.94 

0.40 

12 

0.33 

0.35 

13 

0.24 

0.36 

14 

0.50 

0.65 

15a 

0.73 

0.34 

15b 

0.73 

0.49 

15c 

0.73 

0.52 

15d 

0.62 

0.58 

15e 

0.55 

0.45 

16 

0.39 

0.65 

17 

0.32 

0.47 

18 

0.57 

0.46 

19 

0.28 

0.15 

20 

0.19 

0.34 

21 

0.14 

_b 

22 

0.56 

0.32 

23 

0.29 

0.42 


a All respondents answered correctly, so no biserial correlation can be computed and item removed from final scaling. 
b Item has negative or very low biserial, so is removed from final scaling. 


Table F4 English Language Arts (ELA) 4-6 Item Statistics 


Item/question sequence number 

P-plus value 

Biserial correlation 

1 

0.56 

0.20 

2 

0.64 

0.46 

3 

0.80 

0.41 

4 

0.34 

_a 

5 

0.55 

0.24 

6a 

0.69 

0.38 

6b 

0.65 

0.36 

6c 

0.83 

0.39 

6d 

0.68 

_a 

6e 

0.46 

0.30 

6f 

0.91 

0.29 

7 

0.51 

0.42 

8a 

0.90 

0.29 

8b 

0.69 

0.42 

8c 

0.92 

0.31 

8d 

0.40 

0.35 

9 

0.47 

0.23 

10 

0.34 

0.46 

11a 

0.34 

_a 

lib 

0.96 

0.39 

11c 

0.89 

0.51 

lid 

0.86 

0.45 

lie 

0.71 

0.43 

Ilf 

0.50 

0.58 

12a 

0.52 

0.30 

12b 

0.33 

0.14 
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Table F4 Continued 


Item/question sequence number 

P-plus value 

Biserial correlation 

12c 

0.89 

0.32 

12d 

0.45 

_a 

12e 

0.64 

0.20 

13 

0.49 

0.16 

14 

0.92 

0.40 

15 

0.86 

0.66 

16a 

0.91 

0.16 

16b 

0.86 

0.11 

16c 

0.79 

0.31 

16d 

0.57 

0.50 

16e 

0.76 

0.31 

17 

0.27 

_a 

18 

Constructed-response item 


19a 

0.82 

0.25 

19b 

0.93 

0.44 

19c 

0.44 

0.48 

19d 

0.81 

0.53 

20 

0.64 

0.40 

21 

0.66 

0.29 

22 

Constructed-response item 


23a 

0.44 

0.26 

23b 

0.67 

0.53 

23c 

0.86 

0.14 

23d 

0.80 

0.22 

24 

0.74 

0.55 

25 

0.54 

0.37 

26 

0.79 

0.67 

a Item has negative or very low biserial, 

so is removed from final scaling. 



Table F5 English Language Arts (ELA) 7-9 Item Statistics 

Item/question sequence number P-plus value 

Biserial correlation 

1 

0.20 

0.27 

2 

0.71 

0.41 

3a 

0.88 

0.00 

3b 

0.11 

0.11 

3c 

0.96 

0.32 

3d 

0.69 

0.55 

4 

0.31 

0.18 

5 

0.50 

0.28 

6 

0.52 

0.47 

7a 

0.99 

0.45 

7b 

0.41 

0.39 

7c 

0.80 

0.38 

7d 

0.92 

0.42 

8 

0.69 

0.40 

9 

0.68 

0.15 

10 

0.61 

0.42 

11a 

0.41 

0.19 

lib 

0.94 

0.18 

11c 

0.91 

0.57 

lid 

0.84 

0.38 

lie 

0.76 

0.43 

Ilf 

0.62 

0.59 

12 

Constructed-response item 
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Table F5 Continued 


Item/question sequence number 

P-plus value 

Biserial correlation 

13 

0.79 

0.39 

14 

0.31 

0.10 

15 

0.63 

0.50 

16a 

0.69 

0.47 

16b 

0.46 

0.35 

16c 

0.87 

0.59 

17 

0.67 

0.48 

18 

Constructed-response item 


19a 

0.84 

0.28 

19b 

0.97 

0.49 

19c 

0.52 

0.45 

19d 

0.85 

0.55 

20 

0.72 

0.32 

21a 

0.70 

0.08 

21b 

0.47 

0.10 

21c 

0.16 

_a 

22 

0.46 

0.39 

23a 

0.46 

0.16 

23b 

0.72 

0.31 

23c 

0.87 

_a 

23d 

0.85 

0.03 

24a 

0.88 

0.13 

24b 

0.75 

0.48 

24c 

0.61 

_a 

24d 

0.91 

0.29 

25 

0.68 

0.54 

26 

0.82 

0.51 


a Item has negative or very low biserial, so is removed from final scaling. 
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